- Published
- Updated
Turkish characters, investigator comments and international naming conventions can be corrupted by incorrect encoding. Even readable text may compare or sort differently from the original record, creating errors that a visual spot check misses.
Preserve the meaning of data
Specify the source and destination encoding, normalisation behaviour and permitted symbols. Case conversion deserves particular care in identifier-like fields because letter relationships are not identical in every language. Visual similarity does not demonstrate byte equality or preserved clinical meaning. Keep the original representation accessible when exact text matters to interpretation or reconciliation.
Worked example
For example, dotted and dotless letters in a Turkish device description collapse to one form during transfer. Search results then group two different descriptions as if they were one record. The team tests transfer, read-back and searching together with realistic multilingual examples, while separately investigating the effect on records processed previously.
Review checks
- Include Turkish letters and combined Unicode characters in test examples.
- Read exported text back and compare it with the original source.
- Do not base identity linkage solely on letter normalisation.
Acceptance and boundaries
Acceptance requires preservation of the source meaning and, where necessary, the exact character sequence. Matching personal names still needs authorised local verification; character rules alone do not prove a correct person relationship. Changing an encoding setting will not automatically repair historical corruption. Identify affected fields and a trustworthy correction source before applying any repairs.
Frequently asked questions
- Can corrupted text be corrected manually?
- A reasoned correction can be made against an authoritative source; guessed character substitutions should not manufacture meaning.
- Should all text be normalised?
- A search representation may be normalised, provided its purpose and relationship to the original record stay explicit.