Tools#

This page lists miscellaneous utility components.

Note

For more details about public APIs, refer to medkit.tools.

Save provenance to .dot#

Helper function to generate graphviz-compatible .dot files from provenance data. For more details, refer to medkit.tools.save_prov_to_dot().

HuggingFace utils#

Helper functions for operations using HuggingFace models. For more details, refer to medkit.tools.hf_utils.

mtsamples utils#

Note

For more details about mtsamples data, refer to medkit.tools.mtsamples

The functions provided by this module automatically download mtsamples data into a cache directory before loading / converting into medkit format.

For example, if you want to load the ten first mtsamples text documents:

from medkit.tools.mtsamples import convert_mtsamples_to_medkit, load_mtsamples

docs = load_mtsamples(nb_max=10)

e3c corpus utils#

Note

For more details about e3c corpus data, refer to medkit.tools.e3c_corpus

The E3C corpus is available for download on:

Once downloaded and unzipped, you may :

  • load the data collection into medkit text documents

from pathlib import Path
from medkit.tools.e3c_corpus import load_data_collection

data_collection_layer1 = Path("/tmp/E3C-Corpus-2.0.0/data_collection/French/layer1")

docs = load_data_collection(data_collection_layer1)

  • convert the data collection to medkit text documents.

from pathlib import Path
from medkit.tools.e3c_corpus import convert_data_collection_to_medkit

data_collection = Path("/tmp/E3C-Corpus-2.0.0/data_collection/French")
layers = ["layer1", "layer2", "layer3"]

for layer in layers:
    dir_path = data_collection / layer
    medkit_file = f"medkit_e3c_{layer}.jsonl"
    convert_data_collection_to_medkit(
        dir_path=dir_path, output_file=medkit_file
    )

  • load the annotated data into medkit text documents

from pathlib import Path
from medkit.tools.e3c_corpus import load_data_annotation

data_annotation_layer1 = Path("/tmp/E3C-Corpus-2.0.0/data_annotation/French/layer1")

docs = load_data_annotation(data_annotation_layer1)

  • convert the annotated data to medkit text documents.

from pathlib import Path
from medkit.tools.e3c_corpus import convert_data_annotation_to_medkit

data_annotation = Path("/tmp/E3C-Corpus-2.0.0/data_annotation/French")
layers = ["layer1", "layer2"]

for layer in layers:
    dir_path = data_annotation / layer
    medkit_file = f"medkit_e3c_annotated_{layer}.jsonl"
    convert_data_annotation_to_medkit(
        dir_path=dir_path, output_file=medkit_file
    )