Multilingual support

[Traubert]

As we discussed on Zoom, we at Kielipankki have implemented support for selecting a language for a corpus to be in, and for Sparv annotators to be offered dynamically depending on which language the corpus has. As of now it's quite hard to offer a PR about this that isn't super noisy, because this work involved both removing the old static annotator configurations, fetching annotator information from Sparv (which you have also done), and a bunch of rebranding and UI string rewriting. But the branches this was done in can be viewed as diffs against dev here:

main...CSCfi:Kielipankki-mink-frontend:KP-11146-mink-multilanguage-support

and backend:

spraakbanken/mink-backend@dev...CSCfi:Kielipankki-mink-backend:KP-11290-korp-integrations-fixes

The question is, what's your timeline on adding something similar to SB-Mink, and to what extent are you interested in reusing some aspect of what we did, or does it make more sense for you to have your implementation, which we could then adopt?

[arildm]

Since #217 and #218, we are loading analysis metadata from our Metadata API (https://github.com/spraakbanken/metadata-api), which has a lot in common with annotationMetadata.ts in your branch.

Given that, filtering analyses by language should be a small change.

To make it more usable for you and others, though, we should probably consolidate metadata loading to a pluggable service. And we may need to revise the metadata structure interface. Currently, Mink is using SBX's custom metadata format, but we could convert it to something simpler.

mink-frontend/src/api/analysis.ts

Lines 79 to 95 in e19ca57

	// Manually transcribed from https://github.com/spraakbanken/metadata/blob/main/schema/metadata.json
	export type AnalysisMetadata = {
	id: AnalysisId;
	name: ByLang;
	description: ByLang;
	short_description: ByLang;
	unlisted?: boolean;
	successors?: string[];
	languages: { code: string; name: ByLang };
	analysis_unit?: ByLang;
	in_collections?: string[];
	caveats?: ByLang;
	intended_uses?: ByLang;
	collection?: boolean;
	keywords?: ByLang[];
	task?: string;
	};

Perhaps the scope of this issue could be to just add language filtering. We'll need another issue with a lot more changes to add configurability per instance.

[arildm]

Perhaps the scope of this issue could be to just add language filtering. We'll need another issue with a lot more changes to add configurability per instance.

Configurability issue created: #253