Skip to content

Python API

Everything the command line does is available from Python. Failures raise datanyx.SourceError, with a message that's safe to show: like the command line's, it never quotes the data.

Schemas

import datanyx

print(datanyx.schema_of("orders.csv", "typescript"))          # a schema as text, in one call

For more control, work with the schema as objects. KeyPolicy holds what the command-line options for reading data set: map_paths, keep_names, scale_hints, infer_keys, column_hints, public_columns, views, include, exclude, max_tables, max_rows and delimiter.

from datanyx import KeyPolicy, extract, render

policy = KeyPolicy(map_paths=frozenset({"$.users"}), keep_names=frozenset({"report_2024"}))

schema = extract("customers.json", policy=policy)
print(render(schema, "text"))        # json-schema (default), text, typescript, pydantic, sql or summary
print(schema.notes)                  # what was hidden or left out, as the command line prints to stderr

schema = extract("env:DATABASE_URL", namespace="sales")  # namespace is --schema

Fake rows

from datanyx import extract, synthesize
from datanyx.synth import render_table

tables = synthesize(extract("shop.db"), rows=50, seed=1, messy=True)   # see Synthetic data
print(render_table(tables[0], "csv"))                                 # json, jsonl or csv

Text

import datanyx
from datanyx.text import Anonymizer, PatternDetector, SecretDetector, SpacyDetector, anonymize, anonymize_with_mapping, restore

print(datanyx.anonymize_text("Jessica Whitfield called.", hide=["Acme"]))   # the default detectors

detectors = [PatternDetector(), SecretDetector(), SpacyDetector("en_core_web_lg")]
print(anonymize("Jessica Whitfield called.", detectors, hide_terms=["Acme"]))

clean, mapping = anonymize_with_mapping("Jessica Whitfield called.", detectors)
print(restore(clean, mapping))

shared = Anonymizer(detectors)       # one set of placeholders across many texts
print(shared.anonymize("Ana called."), shared.anonymize("Ana called again."))

Everything else

Module Entry points Page
datanyx.scrub scrub, ScrubPolicy Debugging
datanyx.check load_tables, check_code, format_diagnostics Checking code
datanyx.pseudonymize Pseudonymizer, load_key, pseudonymize_file, Options Pseudonymizing tables
datanyx.text RegexDetector, DictionaryDetector, load_detector_file for your own detectors Detection quality
datanyx.mcp_server create_server, run Using it with an agent