Model¶

dnallm.models.model ¶

DNA Model loading and management utilities.

This module provides functions for downloading, loading, and managing DNA large language models from various sources including Hugging Face Hub, ModelScope, and local storage.

Attributes¶

RANDOM_INIT_DEFAULT_SEED `module-attribute` ¶

RANDOM_INIT_DEFAULT_SEED = 42

CPU-canonical default seed for from-scratch initialization.

Applied via torch.manual_seed immediately before from_config construction; two loads with the same seed produce identical hash tables.

RANDOM_INIT_SUPPORTED_FAMILIES `module-attribute` ¶

RANDOM_INIT_SUPPORTED_FAMILIES = frozenset({'mamba'})

Special model families sanctioned for random_init=True (D-07).

The allowlist gates the special _handle_* families only: a model whose name marks it as one of the special families (evo2, gpn, enformer, ...) is rejected with a ValueError unless its family appears here, because the special handlers construct models through bespoke paths with no from_config equivalent. Generic Auto*-loadable models (including BERT-style and remote-code Mamba checkpoints) are always allowed and need no entry. "mamba" is the proven trust_remote_code from_config member (D-06/A6): Plant DNAMamba models load through the generic AutoModelForCausalLM path, so the entry documents the sanctioned remote-code architecture rather than unlocking a special handler.

Classes¶

DNALLMforSequenceClassification ¶

DNALLMforSequenceClassification(config, custom_model=None)

Bases: PreTrainedModel

An automated wrapper that selects an appropriate pooling strategy based on the underlying model architecture and appends a customizable MLP head for sequence classification or regression tasks.

Methods:¶

from_base_model `classmethod` ¶

from_base_model(
    model_name_or_path,
    config,
    module=None,
    quantization_config=None,
)

Handles weights diffusion when loading a model from a pre-trained base model.

FocalLoss ¶

FocalLoss(alpha=0.25, gamma=2.0, reduction='mean')

Bases: Module

Focal Loss for addressing class imbalance in classification.

Focal Loss down-weights well-classified examples and focuses on hard examples, making it effective for imbalanced datasets.

Reference

Lin, T.Y. et al. "Focal Loss for Dense Object Detection." ICCV 2017.

Parameters:

Name	Type	Description	Default
`alpha`	`float`	Weighting factor for the rare class (default: 0.25).	`0.25`
`gamma`	`float`	Focusing parameter; higher values focus more on hard examples (default: 2.0).	`2.0`
`reduction`	`str`	Reduction to apply to the output: 'none' \| 'mean' \| 'sum'.	`'mean'`

Methods:¶

forward ¶

forward(inputs, targets)

Compute focal loss between inputs and targets.

Parameters:

Name	Type	Description	Default
`inputs`	`Tensor`	Raw model outputs (logits).	required
`targets`	`Tensor`	Ground truth labels.	required

Returns:

Type	Description
`Tensor`	Scalar loss value (or unreduced tensor if reduction='none').

Functions:¶

clear_model_cache ¶

clear_model_cache(source='huggingface')

Remove all the cached models

Parameters:

Name	Type	Description	Default
`source`	`str`	Source to clear model cache from ( 'huggingface', 'modelscope'), default 'huggingface'	`'huggingface'`

download_model ¶

download_model(
    model_name,
    downloader,
    revision=None,
    max_try=10,
    allow_patterns=None,
)

Download a model with retry mechanism for network issues.

In case of network issues, this function will attempt to download the model multiple times before giving up.

Parameters:

Name	Type	Description	Default
`model_name`	`str`	Name of the model to download	required
`downloader`	`Any`	Download function to use (e.g., snapshot_download)	required
`max_try`	`int`	Maximum number of download attempts, default 10	`10`
`allow_patterns`	`list[str] \| None`	Optional glob patterns restricting which snapshot files are fetched (e.g. `[".safetensors", ".json"]`). When `None` (the default) the downloader is called exactly as before -- no `allow_patterns` key is forwarded, so every existing caller behaves byte-identically.	`None`

Returns:

Type	Description
`str`	Path where the model files are stored

Raises:

Type	Description
`ValueError`	If model download fails after all attempts

load_model_and_tokenizer ¶

load_model_and_tokenizer(
    model_name,
    task_config,
    source="local",
    use_mirror=False,
    revision=None,
    custom_tokenizer=None,
    quantization_config=None,
    random_init=False,
    random_init_seed=RANDOM_INIT_DEFAULT_SEED,
)

Load model and tokenizer from either HuggingFace or ModelScope.

This function handles loading of various model types based on the task configuration, including sequence classification, token classification, masked language modeling, and causal language modeling.

Args:
    model_name: Model name or path
    task_config: Task configuration object containing task type and
        label information
            source: Source to load model and tokenizer from (
        'local',
        'huggingface',
        'modelscope'),
        default 'local'
            use_mirror: Whether to use HuggingFace mirror (
        hf-mirror.com),
        default False
    random_init: Build the model from scratch with randomly
        initialized weights (``AutoConfig.from_pretrained`` +
        ``Auto*.from_config``) instead of loading a pretrained
        checkpoint. Only ``config.json`` and tokenizer files are
        fetched — the weight-download path is never invoked. A loud
        "randomly initialized" banner and a per-tensor parameter
        hash table are logged as proof (BASE-01).
    random_init_seed: Seed applied via ``torch.manual_seed``
        immediately before from-scratch construction
        (default RANDOM_INIT_DEFAULT_SEED = 42); identical seeds
        reproduce identical per-tensor hash tables.

Returns:
    Tuple containing (model, tokenizer)

Raises:
    ValueError: If model is not found locally or loading fails;
        if ``random_init=True`` is requested for a special model
        family outside RANDOM_INIT_SUPPORTED_FAMILIES, or combined
        with ``quantization_config`` or a custom ``head_config``

load_preset_model ¶

load_preset_model(model_name, task_config)

Load a preset model and tokenizer based on the task configuration.

This function loads models from the preset model registry, which contains pre-configured models for various DNA analysis tasks.

Parameters:

Name	Type	Description	Default
`model_name`	`str`	Name or path of the model task_config: Task configuration object containing task type and label information	required

Returns:

Type	Description
`tuple[Any, Any] \| int`	Tuple containing (model, tokenizer) if successful, 0 if model not found

Note

If the model is not found in preset models,

    the function will print a warning
        and
    return 0. Use `load_model_and_tokenizer` function for custom model
    loading.

peft_forward_compatiable ¶

peft_forward_compatiable(model)

Convert base model forward to be compatiable with HF

Parameters:

Name	Type	Description	Default
`model`	`Any`	Base model	required

Returns:

Type	Description
`Any`	model with changed forward function

Model¶

dnallm.models.model ¶

Attributes¶

RANDOM_INIT_DEFAULT_SEED module-attribute ¶

RANDOM_INIT_SUPPORTED_FAMILIES module-attribute ¶

Classes¶

DNALLMforSequenceClassification ¶

Methods:¶

from_base_model classmethod ¶

FocalLoss ¶

Methods:¶

forward ¶

Functions:¶

clear_model_cache ¶

download_model ¶

load_model_and_tokenizer ¶

load_preset_model ¶

peft_forward_compatiable ¶

RANDOM_INIT_DEFAULT_SEED `module-attribute` ¶

RANDOM_INIT_SUPPORTED_FAMILIES `module-attribute` ¶

from_base_model `classmethod` ¶