UnicodeEncodeError in Windows

[A-Yucel]

Hi, first of all thanks for this great library. While preparing data for training using trainer.prepare_training_data() , I got this error UnicodeEncodeError: 'charmap' codec can't encode character '\u0131' in position 18: character maps to <undefined>.

What I found is that ragatouille library is trying to write data containing the Turkish character 'ı' (dotless i, Unicode '\u0131') using my system's default Windows encoding (cp1252), which doesn't support this character (this is also the case for some other Turkish characters). The problem was in the function export_training_data from ./data/training_data_processor.py. Simply adding encoding="utf-8" fixed the issue.

./data/training_data_processsor.py, line 265-272:

with open(path / "queries.train.colbert.tsv", "w", encoding="utf-8") as f:
    for query, idx in self.query_map.items():
        query = query.replace("\t", " ").replace("\n", " ")
        f.write(f"{idx}\t{query}\n")
with open(path / "corpus.train.colbert.tsv", "w", encoding="utf-8") as f:
    for document, idx in self.passage_map.items():
        document = document.replace("\t", " ").replace("\n", " ")
        f.write(f"{idx}\t{document}\n")