# Inkling [Inkling](https://huggingface.co/thinkingmachines/Inkling) is a general-purpose multimodal model from [Thinking Machines Lab](https://huggingface.co/thinkingmachines) that accepts text, image, and audio inputs and generates text. It is a 66-layer decoder-only transformer with a sparse mixture-of-experts (MoE) feed-forward backbone — each token is routed to 6 of 256 experts alongside 2 shared experts that are always active — for 975B total parameters with 41B active per token. Image and audio inputs are projected into the language model's embedding space and interleaved with text tokens, so a single checkpoint reasons jointly over all three modalities. You can find the official checkpoints under the [Thinking Machines Lab](https://huggingface.co/thinkingmachines) organization. The example below demonstrates how to generate text based on an image with [Pipeline](/docs/transformers/main/en/main_classes/pipelines#transformers.Pipeline) or the [AutoModel](/docs/transformers/main/en/model_doc/auto#transformers.AutoModel) class. ```python from transformers import pipeline model_id = "thinkingmachines/Inkling-NVFP4" pipe = pipeline("image-text-to-text", model=model_id) image_url = ( "https://huggingface.co/datasets/merve/vl-test-suite/" "resolve/main/pills.jpg" ) messages = [ { "role": "user", "content": [ { "type": "image", "image": image_url, }, { "type": "text", "text": "Do components in this supplement interact with each other?", }, ], }, ] output = pipe( messages, max_new_tokens=2000, return_full_text=False, reasoning_effort="medium", ) output[0]["generated_text"] ``` ```py from transformers import AutoModelForMultimodalLM, AutoProcessor model_id = "thinkingmachines/Inkling-NVFP4" processor = AutoProcessor.from_pretrained(model_id) model = AutoModelForMultimodalLM.from_pretrained( model_id, device_map="auto", ) messages = [ {"role": "system", "content": "You should only answer with a number."}, {"role": "user", "content": "What is 17 * 23?"}, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", reasoning_effort="high", ).to(model.device) output = model.generate(**inputs, max_new_tokens=2000) generated_tokens = output[0][inputs["input_ids"].shape[1] :] print(processor.decode(generated_tokens, skip_special_tokens=False)) ``` ## Notes - Text and image inference: ```py from transformers import AutoModelForMultimodalLM, AutoProcessor model_id = "thinkingmachines/Inkling" processor = AutoProcessor.from_pretrained(model_id) model = AutoModelForMultimodalLM.from_pretrained( model_id, device_map="auto", ) image_url = ( "https://huggingface.co/datasets/merve/vl-test-suite/" "resolve/main/pills.jpg" ) messages = [ { "role": "user", "content": [ { "type": "image", "image": image_url, }, { "type": "text", "text": "Do any of the components in this supplement interact?", }, ], }, ] inputs = processor.apply_chat_template( messages, tokenize=True, add_generation_prompt=True, reasoning_effort="medium", return_dict=True, return_tensors="pt", ).to(model.device) input_len = inputs["input_ids"].shape[-1] outputs = model.generate(**inputs, max_new_tokens=2000) response = processor.decode(outputs[0][input_len:], skip_special_tokens=False) processor.parse_response(response) ``` - Text with audio inference: ```py from transformers import AutoModelForMultimodalLM, AutoProcessor model_id = "thinkingmachines/Inkling" processor = AutoProcessor.from_pretrained(model_id) model = AutoModelForMultimodalLM.from_pretrained( model_id, device_map="auto", ) audio_url = ( "https://huggingface.co/datasets/merve/vl-test-suite/" "resolve/main/example_audio.mp3" ) messages = [ { "role": "user", "content": [ {"type": "text", "text": "Transcribe the following speech to text."}, { "type": "audio", "audio": audio_url, }, ], }, ] inputs = processor.apply_chat_template( messages, tokenize=True, return_dict=True, return_tensors="pt", add_generation_prompt=True, ).to(model.device) input_len = inputs["input_ids"].shape[-1] outputs = model.generate(**inputs, max_new_tokens=512) response = processor.decode(outputs[0][input_len:], skip_special_tokens=False) processor.parse_response(response) ``` - Serving with `transformers serve`: ```shell transformers serve thinkingmachines/Inkling-NVFP4 ``` ```py from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="") completion = client.chat.completions.create( model="thinkingmachines/Inkling-NVFP4", messages=[ { "role": "user", "content": [ {"type": "text", "text": "What is in this image?"}, { "type": "image_url", "image_url": { "url": "https://huggingface.co/datasets/merve/vl-test-suite/resolve/main/pills.jpg" }, }, ], } ], ) print(completion.choices[0].message.content) ``` ## InklingAudioConfig[[transformers.InklingAudioConfig]] #### transformers.InklingAudioConfig[[transformers.InklingAudioConfig]] ```python transformers.InklingAudioConfig(transformers_version: str | None = None, architectures: list[str] | None = None, output_hidden_states: bool | None = False, return_dict: bool | None = True, dtype: typing.Union[str, ForwardRef('torch.dtype'), NoneType] = None, chunk_size_feed_forward: int = 0, is_encoder_decoder: bool = False, id2label: dict[int, str] | dict[str, str] | None = None, label2id: dict[str, int] | dict[str, str] | None = None, problem_type: typing.Optional[typing.Literal['regression', 'single_label_classification', 'multi_label_classification']] = None, n_mel_bins: int = 80, mel_vocab_size: int = 256, text_hidden_size: int = 6144, rms_norm_eps: float = 1e-06, initializer_range: float = 0.02) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/configuration_inkling.py#L201) **Parameters:** n_mel_bins (`int`, *optional*, defaults to 80) : Number of mel-frequency bins per audio frame. mel_vocab_size (`int`, *optional*, defaults to 256) : Number of discrete bins each mel value is quantized into before being embedded. text_hidden_size (`int`, *optional*, defaults to 6144) : Dimensionality the audio embeddings are projected to, matching the text backbone. rms_norm_eps (`float`, *optional*, defaults to `1e-06`) : The epsilon used by the rms normalization layers. initializer_range (`float`, *optional*, defaults to `0.02`) : The standard deviation of the truncated_normal_initializer for initializing all weight matrices. This is the configuration class to store the configuration of a InklingModel. It is used to instantiate a Inkling model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the [thinkingmachines/Inkling](https://huggingface.co/thinkingmachines/Inkling) Configuration objects inherit from [PreTrainedConfig](/docs/transformers/main/en/main_classes/configuration#transformers.PreTrainedConfig) and can be used to control the model outputs. Read the documentation from [PreTrainedConfig](/docs/transformers/main/en/main_classes/configuration#transformers.PreTrainedConfig) for more information. ## InklingConfig[[transformers.InklingConfig]] #### transformers.InklingConfig[[transformers.InklingConfig]] ```python transformers.InklingConfig(transformers_version: str | None = None, architectures: list[str] | None = None, output_hidden_states: bool | None = False, return_dict: bool | None = True, dtype: typing.Union[str, ForwardRef('torch.dtype'), NoneType] = None, chunk_size_feed_forward: int = 0, is_encoder_decoder: bool = False, id2label: dict[int, str] | dict[str, str] | None = None, label2id: dict[str, int] | dict[str, str] | None = None, problem_type: typing.Optional[typing.Literal['regression', 'single_label_classification', 'multi_label_classification']] = None, text_config: transformers.models.inkling.configuration_inkling.InklingTextConfig | dict | None = None, audio_config: transformers.models.inkling.configuration_inkling.InklingAudioConfig | dict | None = None, vision_config: transformers.models.inkling.configuration_inkling.InklingVisionConfig | dict | None = None, image_token_id: int = 200054, audio_token_id: int = 200053, image_bos_token_id: int = 200005, audio_bos_token_id: int = 200020) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/configuration_inkling.py#L254) **Parameters:** text_config (`Union[~models.inkling.configuration_inkling.InklingTextConfig, dict]`, *optional*) : The config object or dictionary of the text backbone. audio_config (`Union[~models.inkling.configuration_inkling.InklingAudioConfig, dict]`, *optional*) : The config object or dictionary of the audio backbone. vision_config (`Union[~models.inkling.configuration_inkling.InklingVisionConfig, dict]`, *optional*) : The config object or dictionary of the vision backbone. image_token_id (`int`, *optional*, defaults to `200054`) : The image token index used as a placeholder for input images. audio_token_id (`int`, *optional*, defaults to `200053`) : The audio token index used as a placeholder for input audio. image_bos_token_id (`int`, *optional*, defaults to 200005) : The beginning-of-image token index used to mark the start of image spans. audio_bos_token_id (`int`, *optional*, defaults to 200020) : The beginning-of-audio token index used to mark the start of audio spans. Top-level multimodal config (`InklingMMConfig` in the SGLang source). ## InklingTextConfig[[transformers.InklingTextConfig]] #### transformers.InklingTextConfig[[transformers.InklingTextConfig]] ```python transformers.InklingTextConfig(transformers_version: str | None = None, architectures: list[str] | None = None, output_hidden_states: bool | None = False, return_dict: bool | None = True, dtype: typing.Union[str, ForwardRef('torch.dtype'), NoneType] = None, chunk_size_feed_forward: int = 0, is_encoder_decoder: bool = False, id2label: dict[int, str] | dict[str, str] | None = None, label2id: dict[str, int] | dict[str, str] | None = None, problem_type: typing.Optional[typing.Literal['regression', 'single_label_classification', 'multi_label_classification']] = None, vocab_size: int = 201024, unpadded_vocab_size: int | None = None, hidden_size: int = 6144, num_hidden_layers: int = 66, num_attention_heads: int = 64, num_key_value_heads: int = 8, head_dim: int = 128, swa_num_attention_heads: int = 64, swa_num_key_value_heads: int = 16, swa_head_dim: int = 128, sliding_window_size: int = 512, d_rel: int = 16, rel_extent: int = 1024, log_scaling_n_floor: int | None = None, log_scaling_alpha: float = 0.1, local_layer_ids: list[int] | None = None, layer_types: list[str] | None = None, max_position_embeddings: int = 131072, rms_norm_eps: float = 1e-06, conv_kernel_size: int = 4, mlp_layer_types: list[str] | None = None, intermediate_size: int = 24576, hidden_act: str = 'silu', moe_intermediate_size: int = 3072, n_routed_experts: int = 256, num_experts_per_tok: int = 6, n_shared_experts: int = 2, shared_expert_sink: bool = True, route_scale: float = 8.0, logits_mup_width_multiplier: float = 24.0, rms_norm_eps_moe_gate: float = 1e-06, attention_dropout: float = 0.0, initializer_range: float = 0.02, pad_token_id: int | None = None, bos_token_id: int | None = 1, eos_token_id: int | None = 2, num_mtp_layers: int | None = None, chain_hidden_post_norm: bool = False, mtp_hidden_states_first: bool = True, mtp_local_layer_ids: list[int] | None = None) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/configuration_inkling.py#L30) **Parameters:** vocab_size (`int`, *optional*, defaults to `201024`) : Vocabulary size of the model. Defines the number of different tokens that can be represented by the `input_ids`. unpadded_vocab_size (`int`, *optional*, defaults to `None`) : Number of rows the checkpoint's unembedding matrix actually holds when the head is not padded to `vocab_size`. Logits beyond it are dropped. If `None`, the head is not padded. hidden_size (`int`, *optional*, defaults to `6144`) : Dimension of the hidden representations. num_hidden_layers (`int`, *optional*, defaults to `66`) : Number of hidden layers in the Transformer decoder. num_attention_heads (`int`, *optional*, defaults to `64`) : Number of attention heads for each attention layer in the Transformer decoder. num_key_value_heads (`int`, *optional*, defaults to `8`) : This is the number of key_value heads that should be used to implement Grouped Query Attention. If `num_key_value_heads=num_attention_heads`, the model will use Multi Head Attention (MHA), if `num_key_value_heads=1` the model will use Multi Query Attention (MQA) otherwise GQA is used. When converting a multi-head checkpoint to a GQA checkpoint, each group key and value head should be constructed by meanpooling all the original heads within that group. For more details, check out [this paper](https://huggingface.co/papers/2305.13245). If it is not specified, will default to `num_attention_heads`. head_dim (`int`, *optional*, defaults to `128`) : The attention head dimension. If None, it will default to hidden_size // num_attention_heads swa_num_attention_heads (`int`, *optional*, defaults to 64) : Number of attention heads in the sliding-window layers. swa_num_key_value_heads (`int`, *optional*, defaults to 16) : Number of key/value heads in the sliding-window layers. swa_head_dim (`int`, *optional*, defaults to 128) : Dimension of query and key heads in the sliding-window layers. sliding_window_size (`int`, *optional*, defaults to 512) : Size of the sliding attention window used by layers whose `layer_types` entry is `"hybrid_sliding"`. d_rel (`int`, *optional*, defaults to 16) : Per-head dimension of the relative states that are mixed into the relative position bias. rel_extent (`int`, *optional*, defaults to 1024) : Backward distance, in tokens, over which the relative position bias is applied. The bias is zero beyond it. log_scaling_n_floor (`int`, *optional*, defaults to `None`) : Position from which logits start being scaled up logarithmically in the full-attention layers. If `None`, the scaling is disabled. log_scaling_alpha (`float`, *optional*, defaults to 0.1) : Strength of the logarithmic logit scaling controlled by `log_scaling_n_floor`. local_layer_ids (`list[int]`, *optional*, defaults to `None`) : Indices of the layers using sliding window attention. Used to derive `layer_types` when it is not provided. If `None`, every layer whose index is not a multiple of 6 uses sliding window attention. layer_types (`list[str]`, *optional*) : A list that explicitly maps each layer index with its layer type. If not provided, it will be automatically generated based on config values. max_position_embeddings (`int`, *optional*, defaults to `131072`) : The maximum sequence length that this model might ever be used with. rms_norm_eps (`float`, *optional*, defaults to `1e-06`) : The epsilon used by the rms normalization layers. conv_kernel_size (`int`, *optional*, defaults to `4`) : The size of the convolutional kernel. mlp_layer_types (`list[str]`, *optional*, defaults to `None`) : MLP type pattern for each layer (`"dense"` or `"sparse"`). If `None`, every layer is sparse. intermediate_size (`int`, *optional*, defaults to `24576`) : Dimension of the MLP representations. hidden_act (`str`, *optional*, defaults to `silu`) : The non-linear activation function (function or string) in the decoder. For example, `"gelu"`, `"relu"`, `"silu"`, etc. moe_intermediate_size (`int`, *optional*, defaults to `3072`) : Intermediate size of the routed expert MLPs. n_routed_experts (`int`, *optional*, defaults to `256`) : Number of routed experts. num_experts_per_tok (`int`, *optional*, defaults to `6`) : Number of experts to route each token to. This is the top-k value for the token-choice routing. n_shared_experts (`int`, *optional*, defaults to `2`) : Number of shared experts. shared_expert_sink (`bool`, *optional*, defaults to `True`) : Whether the router scores the shared experts alongside the routed ones, so that they act as a sink in the softmax over expert weights. route_scale (`float`, *optional*, defaults to `8.0`) : Scaling factor or routed experts. logits_mup_width_multiplier (`float`, *optional*, defaults to 24.0) : muP width multiplier the final hidden states are divided by before the language modeling head. rms_norm_eps_moe_gate (`float`, *optional*, defaults to 1e-6) : Epsilon of the RMS normalization applied inside the mixture-of-experts router. attention_dropout (`float`, *optional*, defaults to `0.0`) : The dropout ratio for the attention probabilities. initializer_range (`float`, *optional*, defaults to `0.02`) : The standard deviation of the truncated_normal_initializer for initializing all weight matrices. pad_token_id (`int`, *optional*) : Token id used for padding in the vocabulary. bos_token_id (`int`, *optional*, defaults to `1`) : Token id used for beginning-of-stream in the vocabulary. eos_token_id (`int`, *optional*, defaults to `2`) : Token id used for end-of-stream in the vocabulary. num_mtp_layers (`int`, *optional*, defaults to `None`) : Number of multi-token-prediction layers. If `None`, multi-token prediction is disabled. chain_hidden_post_norm (`bool`, *optional*, defaults to `False`) : Whether the hidden states chained between multi-token-prediction layers are normalized after each layer. mtp_hidden_states_first (`bool`, *optional*, defaults to `True`) : Whether the hidden states come before the token embeddings when the two are concatenated as the input of a multi-token-prediction layer. mtp_local_layer_ids (`list[int]`, *optional*, defaults to `None`) : Indices of the multi-token-prediction layers using sliding window attention. If `None`, every multi-token-prediction layer uses full attention. This is the configuration class to store the configuration of a InklingModel. It is used to instantiate a Inkling model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the [thinkingmachines/Inkling](https://huggingface.co/thinkingmachines/Inkling) Configuration objects inherit from [PreTrainedConfig](/docs/transformers/main/en/main_classes/configuration#transformers.PreTrainedConfig) and can be used to control the model outputs. Read the documentation from [PreTrainedConfig](/docs/transformers/main/en/main_classes/configuration#transformers.PreTrainedConfig) for more information. ## InklingVisionConfig[[transformers.InklingVisionConfig]] #### transformers.InklingVisionConfig[[transformers.InklingVisionConfig]] ```python transformers.InklingVisionConfig(transformers_version: str | None = None, architectures: list[str] | None = None, output_hidden_states: bool | None = False, return_dict: bool | None = True, dtype: typing.Union[str, ForwardRef('torch.dtype'), NoneType] = None, chunk_size_feed_forward: int = 0, is_encoder_decoder: bool = False, id2label: dict[int, str] | dict[str, str] | None = None, label2id: dict[str, int] | dict[str, str] | None = None, problem_type: typing.Optional[typing.Literal['regression', 'single_label_classification', 'multi_label_classification']] = None, text_hidden_size: int = 6144, patch_size: int = 40, temporal_patch_size: int = 2, num_channels: int = 3, hidden_size: int = 1024, num_hidden_layers: int = 24, num_attention_heads: int = 16, rms_norm_eps: float = 1e-06, initializer_range: float = 0.02) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/configuration_inkling.py#L228) **Parameters:** text_hidden_size (`int`, *optional*, defaults to 6144) : Dimensionality the vision features are projected to by the last encoder layer, matching the text backbone. patch_size (`int`, *optional*, defaults to `40`) : The size (resolution) of each patch. temporal_patch_size (`int`, *optional*, defaults to `2`) : Temporal patch size used in the 3D patch embedding for video inputs. num_channels (`int`, *optional*, defaults to `3`) : The number of input channels. hidden_size (`int`, *optional*, defaults to `1024`) : Dimension of the hidden representations. num_hidden_layers (`int`, *optional*, defaults to `24`) : Number of hidden layers in the Transformer decoder. num_attention_heads (`int`, *optional*, defaults to `16`) : Number of attention heads for each attention layer in the Transformer decoder. rms_norm_eps (`float`, *optional*, defaults to `1e-06`) : The epsilon used by the rms normalization layers. initializer_range (`float`, *optional*, defaults to `0.02`) : The standard deviation of the truncated_normal_initializer for initializing all weight matrices. This is the configuration class to store the configuration of a InklingModel. It is used to instantiate a Inkling model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the [thinkingmachines/Inkling](https://huggingface.co/thinkingmachines/Inkling) Configuration objects inherit from [PreTrainedConfig](/docs/transformers/main/en/main_classes/configuration#transformers.PreTrainedConfig) and can be used to control the model outputs. Read the documentation from [PreTrainedConfig](/docs/transformers/main/en/main_classes/configuration#transformers.PreTrainedConfig) for more information. ## InklingAudioModel[[transformers.InklingAudioModel]] #### transformers.InklingAudioModel[[transformers.InklingAudioModel]] ```python transformers.InklingAudioModel(config: InklingAudioConfig) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/modeling_inkling.py#L820) **Parameters:** config ([InklingAudioConfig](/docs/transformers/main/en/model_doc/inkling#transformers.InklingAudioConfig)) : Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [from_pretrained()](/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) method to load the model weights. The bare Inkling Model outputting raw hidden-states without any specific head on top. This model inherits from [PreTrainedModel](/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel). Check the superclass documentation for the generic methods the library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads etc.) This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. #### forward[[transformers.InklingAudioModel.forward]] ```python forward(audio_input_ids: Tensor, **kwargs) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/modeling_inkling.py#L826) **Parameters:** audio_input_ids (`torch.Tensor` of shape `(num_audios, max_num_frames, n_mel_bins)`) : Mel-spectrogram frames of the input audios. **Returns:** [BaseModelOutputWithPooling](/docs/transformers/main/en/main_classes/output#transformers.modeling_outputs.BaseModelOutputWithPooling) or `tuple(torch.FloatTensor)` A [BaseModelOutputWithPooling](/docs/transformers/main/en/main_classes/output#transformers.modeling_outputs.BaseModelOutputWithPooling) or a tuple of `torch.FloatTensor` (if `return_dict=False` is passed or when `config.return_dict=False`) comprising various elements depending on the configuration ([InklingConfig](/docs/transformers/main/en/model_doc/inkling#transformers.InklingConfig)) and inputs. The [InklingAudioModel](/docs/transformers/main/en/model_doc/inkling#transformers.InklingAudioModel) forward method, overrides the `__call__` special method. Although the recipe for forward pass needs to be defined within this function, one should call the `Module` instance afterwards instead of this since the former takes care of running the pre and post processing steps while the latter silently ignores them. - **last_hidden_state** (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`) -- Sequence of hidden-states at the output of the last layer of the model. - **pooler_output** (`torch.FloatTensor` of shape `(batch_size, hidden_size)`) -- Last layer hidden-state of the first token of the sequence (classification token) after further processing through the layers used for the auxiliary pretraining task. E.g. for BERT-family of models, this returns the classification token after processing through a linear layer and a tanh activation function. The linear layer weights are trained from the next sentence prediction (classification) objective during pretraining. - **hidden_states** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`) -- Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. - **attentions** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`) -- Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. ## InklingForCausalLM[[transformers.InklingForCausalLM]] #### transformers.InklingForCausalLM[[transformers.InklingForCausalLM]] ```python transformers.InklingForCausalLM(config: InklingTextConfig) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/modeling_inkling.py#L726) **Parameters:** config ([InklingTextConfig](/docs/transformers/main/en/model_doc/inkling#transformers.InklingTextConfig)) : Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [from_pretrained()](/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) method to load the model weights. The Inkling Model for causal language modeling. This model inherits from [PreTrainedModel](/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel). Check the superclass documentation for the generic methods the library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads etc.) This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. #### forward[[transformers.InklingForCausalLM.forward]] ```python forward(input_ids: typing.Optional[torch.LongTensor] = None, attention_mask: typing.Optional[torch.Tensor] = None, position_ids: typing.Optional[torch.LongTensor] = None, past_key_values: transformers.cache_utils.Cache | None = None, inputs_embeds: typing.Optional[torch.FloatTensor] = None, labels: typing.Optional[torch.LongTensor] = None, use_cache: bool | None = None, logits_to_keep: typing.Union[int, torch.Tensor] = 0, **kwargs: Unpack) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/modeling_inkling.py#L743) **Parameters:** input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) : Indices of input sequence tokens in the vocabulary. Padding will be ignored by default. Indices can be obtained using [AutoTokenizer](/docs/transformers/main/en/model_doc/auto#transformers.AutoTokenizer). See [PreTrainedTokenizer.encode()](/docs/transformers/main/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.encode) and [PreTrainedTokenizer.__call__()](/docs/transformers/main/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.__call__) for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*) : Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) position_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) : Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. [What are position IDs?](../glossary#position-ids) past_key_values (`~cache_utils.Cache`, *optional*) : Pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used to speed up sequential decoding. This typically consists in the `past_key_values` returned by the model at a previous stage of decoding, when `use_cache=True` or `config.use_cache=True`. Only [Cache](/docs/transformers/main/en/internal/generation_utils#transformers.Cache) instance is allowed as input, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). If no `past_key_values` are passed, [DynamicCache](/docs/transformers/main/en/internal/generation_utils#transformers.DynamicCache) will be initialized by default. The model will output the same cache format that is fed as input. If `past_key_values` are used, the user is expected to input only unprocessed `input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, unprocessed_length)` instead of all `input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*) : Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) : Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. use_cache (`bool`, *optional*) : If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). logits_to_keep (`Union[int, torch.Tensor]`, *optional*, defaults to `0`) : If an `int`, compute logits for the last `logits_to_keep` tokens. If `0`, calculate logits for all `input_ids` (special case). Only last token logits are needed for generation, and calculating them only for that token can save memory, which becomes pretty significant for long sequences or large vocabulary size. If a `torch.Tensor`, must be 1D corresponding to the indices to keep in the sequence length dimension. This is useful when using packed tensor format (single dimension for batch and sequence length). **Returns:** `InklingCausalLMOutputWithPast` or `tuple(torch.FloatTensor)` A `InklingCausalLMOutputWithPast` or a tuple of `torch.FloatTensor` (if `return_dict=False` is passed or when `config.return_dict=False`) comprising various elements depending on the configuration ([InklingConfig](/docs/transformers/main/en/model_doc/inkling#transformers.InklingConfig)) and inputs. The [InklingForCausalLM](/docs/transformers/main/en/model_doc/inkling#transformers.InklingForCausalLM) forward method, overrides the `__call__` special method. Although the recipe for forward pass needs to be defined within this function, one should call the `Module` instance afterwards instead of this since the former takes care of running the pre and post processing steps while the latter silently ignores them. - **loss** (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided) -- Language modeling loss (for next-token prediction). - **logits** (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.text_config.vocab_size)`) -- Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax). - **past_key_values** (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`) -- It is a [Cache](/docs/transformers/main/en/internal/generation_utils#transformers.Cache) instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. - **hidden_states** (`tuple[torch.FloatTensor]`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`) -- Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. - **attentions** (`tuple[torch.FloatTensor]`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`) -- Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. - **image_hidden_states** (`torch.FloatTensor`, *optional*) -- A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder after projecting last hidden state. Example: ```python >>> from transformers import AutoTokenizer, InklingForCausalLM >>> model = InklingForCausalLM.from_pretrained("thinkingmachines/Inkling-NVFP4") >>> tokenizer = AutoTokenizer.from_pretrained("thinkingmachines/Inkling-NVFP4") >>> prompt = "What is your favorite condiment?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(**inputs, max_new_tokens=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "What is your favorite condiment?" ``` ## InklingForConditionalGeneration[[transformers.InklingForConditionalGeneration]] #### transformers.InklingForConditionalGeneration[[transformers.InklingForConditionalGeneration]] ```python transformers.InklingForConditionalGeneration(config: InklingConfig) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/modeling_inkling.py#L1189) **Parameters:** config ([InklingConfig](/docs/transformers/main/en/model_doc/inkling#transformers.InklingConfig)) : Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [from_pretrained()](/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) method to load the model weights. The Base Inkling model which consists of a vision backbone and a language model without language modeling head., This model inherits from [PreTrainedModel](/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel). Check the superclass documentation for the generic methods the library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads etc.) This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. #### forward[[transformers.InklingForConditionalGeneration.forward]] ```python forward(input_ids: typing.Optional[torch.LongTensor] = None, pixel_values: typing.Optional[torch.FloatTensor] = None, attention_mask: typing.Optional[torch.Tensor] = None, position_ids: typing.Optional[torch.LongTensor] = None, past_key_values: transformers.cache_utils.Cache | None = None, audio_input_ids: typing.Optional[torch.LongTensor] = None, audio_input_ids_mask: typing.Optional[torch.Tensor] = None, inputs_embeds: typing.Optional[torch.FloatTensor] = None, labels: typing.Optional[torch.LongTensor] = None, use_cache: bool | None = None, logits_to_keep: typing.Union[int, torch.Tensor] = 0, **kwargs: Unpack) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/modeling_inkling.py#L1209) **Parameters:** input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) : Indices of input sequence tokens in the vocabulary. Padding will be ignored by default. Indices can be obtained using [AutoTokenizer](/docs/transformers/main/en/model_doc/auto#transformers.AutoTokenizer). See [PreTrainedTokenizer.encode()](/docs/transformers/main/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.encode) and [PreTrainedTokenizer.__call__()](/docs/transformers/main/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.__call__) for details. [What are input IDs?](../glossary#input-ids) pixel_values (`torch.FloatTensor` of shape `(batch_size, num_channels, image_size, image_size)`, *optional*) : The tensors corresponding to the input images. Pixel values can be obtained using `image_processor_class`. See `image_processor_class.__call__` for details (`processor_class` uses `image_processor_class` for processing images). attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*) : Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) position_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) : Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. [What are position IDs?](../glossary#position-ids) past_key_values (`~cache_utils.Cache`, *optional*) : Pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used to speed up sequential decoding. This typically consists in the `past_key_values` returned by the model at a previous stage of decoding, when `use_cache=True` or `config.use_cache=True`. Only [Cache](/docs/transformers/main/en/internal/generation_utils#transformers.Cache) instance is allowed as input, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). If no `past_key_values` are passed, [DynamicCache](/docs/transformers/main/en/internal/generation_utils#transformers.DynamicCache) will be initialized by default. The model will output the same cache format that is fed as input. If `past_key_values` are used, the user is expected to input only unprocessed `input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, unprocessed_length)` instead of all `input_ids` of shape `(batch_size, sequence_length)`. audio_input_ids (`torch.LongTensor` of shape `(num_audios, max_num_frames, n_mel_bins)`, *optional*) : Batch of (padded) discretized dMel bin tokens produced by [InklingProcessor](/docs/transformers/main/en/model_doc/inkling#transformers.InklingProcessor). audio_input_ids_mask (`torch.Tensor` of shape `(num_audios, max_num_frames)`, *optional*) : Mask marking valid (non-padding) audio frames in `audio_input_ids`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*) : Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) : Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.text_config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.text_config.vocab_size]`. use_cache (`bool`, *optional*) : If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). logits_to_keep (`Union[int, torch.Tensor]`, *optional*, defaults to `0`) : If an `int`, compute logits for the last `logits_to_keep` tokens. If `0`, calculate logits for all `input_ids` (special case). Only last token logits are needed for generation, and calculating them only for that token can save memory, which becomes pretty significant for long sequences or large vocabulary size. If a `torch.Tensor`, must be 1D corresponding to the indices to keep in the sequence length dimension. This is useful when using packed tensor format (single dimension for batch and sequence length). **Returns:** `InklingCausalLMOutputWithPast` or `tuple(torch.FloatTensor)` A `InklingCausalLMOutputWithPast` or a tuple of `torch.FloatTensor` (if `return_dict=False` is passed or when `config.return_dict=False`) comprising various elements depending on the configuration ([InklingConfig](/docs/transformers/main/en/model_doc/inkling#transformers.InklingConfig)) and inputs. The [InklingForConditionalGeneration](/docs/transformers/main/en/model_doc/inkling#transformers.InklingForConditionalGeneration) forward method, overrides the `__call__` special method. Although the recipe for forward pass needs to be defined within this function, one should call the `Module` instance afterwards instead of this since the former takes care of running the pre and post processing steps while the latter silently ignores them. - **loss** (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided) -- Language modeling loss (for next-token prediction). - **logits** (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.text_config.vocab_size)`) -- Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax). - **past_key_values** (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`) -- It is a [Cache](/docs/transformers/main/en/internal/generation_utils#transformers.Cache) instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. - **hidden_states** (`tuple[torch.FloatTensor]`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`) -- Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. - **attentions** (`tuple[torch.FloatTensor]`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`) -- Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. - **image_hidden_states** (`torch.FloatTensor`, *optional*) -- A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder after projecting last hidden state. Example: ```python >>> from PIL import Image >>> import httpx >>> from io import BytesIO >>> from transformers import AutoProcessor, InklingForConditionalGeneration >>> model = InklingForConditionalGeneration.from_pretrained("thinkingmachines/Inkling-NVFP4") >>> processor = AutoProcessor.from_pretrained("thinkingmachines/Inkling-NVFP4") >>> messages = [ ... { ... "role": "system", ... "content": [ ... {"type": "text", "text": "You are a helpful assistant."} ... ] ... }, ... { ... "role": "user", "content": [ ... {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg"}, ... {"type": "text", "text": "Where is the cat standing?"}, ... ] ... }, ... ] >>> inputs = processor.apply_chat_template( ... messages, ... tokenize=True, ... return_dict=True, ... return_tensors="pt", ... add_generation_prompt=True ... ) >>> # Generate >>> generate_ids = model.generate(**inputs) >>> processor.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "user\nYou are a helpful assistant.\n\n\n\n\n\nWhere is the cat standing?\nmodel\nBased on the image, the cat is standing in a snowy area, likely outdoors. It appears to" ``` #### get_image_features[[transformers.InklingForConditionalGeneration.get_image_features]] ```python get_image_features(pixel_values: FloatTensor, **kwargs: Unpack) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/modeling_inkling.py#L1205) **Parameters:** pixel_values (`torch.FloatTensor` of shape `(batch_size, num_channels, image_size, image_size)`) : The tensors corresponding to the input images. Pixel values can be obtained using `image_processor_class`. See `image_processor_class.__call__` for details (`processor_class` uses `image_processor_class` for processing images). Example: ```python >>> from PIL import Image >>> from transformers import AutoProcessor, InklingForConditionalGeneration >>> model = InklingForConditionalGeneration.from_pretrained("None") >>> processor = AutoProcessor.from_pretrained("None") >>> messages = [ ... { ... "role": "user", "content": [ ... {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg"}, ... {"type": "text", "text": "Where is the cat standing?"}, ... ] ... }, ... ] >>> inputs = processor.apply_chat_template( ... messages, ... tokenize=True, ... return_dict=True, ... return_tensors="pt", ... add_generation_prompt=True ... ) >>> # Generate >>> generate_ids = model.generate(**inputs) >>> processor.batch_decode(generate_ids, skip_special_tokens=True)[0] ``` ## InklingModel[[transformers.InklingModel]] #### transformers.InklingModel[[transformers.InklingModel]] ```python transformers.InklingModel(config: InklingConfig) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/modeling_inkling.py#L1009) **Parameters:** config ([InklingConfig](/docs/transformers/main/en/model_doc/inkling#transformers.InklingConfig)) : Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [from_pretrained()](/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) method to load the model weights. The Base Inkling model which consists of a vision backbone and a language model without language modeling head., This model inherits from [PreTrainedModel](/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel). Check the superclass documentation for the generic methods the library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads etc.) This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. #### forward[[transformers.InklingModel.forward]] ```python forward(input_ids: typing.Optional[torch.LongTensor] = None, pixel_values: typing.Optional[torch.FloatTensor] = None, audio_input_ids: typing.Optional[torch.LongTensor] = None, audio_input_ids_mask: typing.Optional[torch.Tensor] = None, attention_mask: typing.Optional[torch.Tensor] = None, position_ids: typing.Optional[torch.LongTensor] = None, past_key_values: transformers.cache_utils.Cache | None = None, token_type_ids: typing.Optional[torch.LongTensor] = None, inputs_embeds: typing.Optional[torch.FloatTensor] = None, labels: typing.Optional[torch.LongTensor] = None, use_cache: bool | None = None, **lm_kwargs: Unpack) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/modeling_inkling.py#L1075) **Parameters:** input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) : Indices of input sequence tokens in the vocabulary. Padding will be ignored by default. Indices can be obtained using [AutoTokenizer](/docs/transformers/main/en/model_doc/auto#transformers.AutoTokenizer). See [PreTrainedTokenizer.encode()](/docs/transformers/main/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.encode) and [PreTrainedTokenizer.__call__()](/docs/transformers/main/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.__call__) for details. [What are input IDs?](../glossary#input-ids) pixel_values (`torch.FloatTensor` of shape `(batch_size, num_channels, image_size, image_size)`, *optional*) : The tensors corresponding to the input images. Pixel values can be obtained using `image_processor_class`. See `image_processor_class.__call__` for details (`processor_class` uses `image_processor_class` for processing images). audio_input_ids (`torch.LongTensor` of shape `(num_audios, max_num_frames, n_mel_bins)`, *optional*) : Batch of (padded) discretized dMel bin tokens produced by [InklingProcessor](/docs/transformers/main/en/model_doc/inkling#transformers.InklingProcessor). audio_input_ids_mask (`torch.Tensor` of shape `(num_audios, max_num_frames)`, *optional*) : Mask marking valid (non-padding) audio frames in `audio_input_ids`. attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*) : Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) position_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) : Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. [What are position IDs?](../glossary#position-ids) past_key_values (`~cache_utils.Cache`, *optional*) : Pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used to speed up sequential decoding. This typically consists in the `past_key_values` returned by the model at a previous stage of decoding, when `use_cache=True` or `config.use_cache=True`. Only [Cache](/docs/transformers/main/en/internal/generation_utils#transformers.Cache) instance is allowed as input, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). If no `past_key_values` are passed, [DynamicCache](/docs/transformers/main/en/internal/generation_utils#transformers.DynamicCache) will be initialized by default. The model will output the same cache format that is fed as input. If `past_key_values` are used, the user is expected to input only unprocessed `input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, unprocessed_length)` instead of all `input_ids` of shape `(batch_size, sequence_length)`. token_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) : Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0, 1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. [What are token type IDs?](../glossary#token-type-ids) inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*) : Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) : Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.text_config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.text_config.vocab_size]`. use_cache (`bool`, *optional*) : If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). **Returns:** `InklingModelOutputWithPast` or `tuple(torch.FloatTensor)` A `InklingModelOutputWithPast` or a tuple of `torch.FloatTensor` (if `return_dict=False` is passed or when `config.return_dict=False`) comprising various elements depending on the configuration ([InklingConfig](/docs/transformers/main/en/model_doc/inkling#transformers.InklingConfig)) and inputs. The [InklingModel](/docs/transformers/main/en/model_doc/inkling#transformers.InklingModel) forward method, overrides the `__call__` special method. Although the recipe for forward pass needs to be defined within this function, one should call the `Module` instance afterwards instead of this since the former takes care of running the pre and post processing steps while the latter silently ignores them. - **last_hidden_state** (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`) -- Sequence of hidden-states at the output of the last layer of the model. If `past_key_values` is used only the last hidden-state of the sequences of shape `(batch_size, 1, hidden_size)` is output. - **past_key_values** (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`) -- It is a [Cache](/docs/transformers/main/en/internal/generation_utils#transformers.Cache) instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). Contains pre-computed hidden-states (key and values in the self-attention blocks and optionally if `config.is_encoder_decoder=True` in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. - **hidden_states** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`) -- Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. - **attentions** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`) -- Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. - **image_hidden_states** (`torch.FloatTensor`, *optional*) -- A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder and after projecting the last hidden state. Example: ```python >>> from PIL import Image >>> import httpx >>> from io import BytesIO >>> from transformers import AutoProcessor, InklingForConditionalGeneration >>> model = InklingForConditionalGeneration.from_pretrained("thinkingmachines/Inkling-NVFP4") >>> processor = AutoProcessor.from_pretrained("thinkingmachines/Inkling-NVFP4") >>> prompt = "Where is the cat standing?" >>> url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg" >>> with httpx.stream("GET", url) as response: ... image = Image.open(BytesIO(response.read())) >>> inputs = processor(images=image, text=prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(**inputs,) >>> processor.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Where is the cat standing?\nsnow" ``` ## InklingPreTrainedModel[[transformers.InklingPreTrainedModel]] #### transformers.InklingPreTrainedModel[[transformers.InklingPreTrainedModel]] ```python transformers.InklingPreTrainedModel(config: PreTrainedConfig, *inputs, **kwargs) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/modeling_inkling.py#L596) **Parameters:** config ([PreTrainedConfig](/docs/transformers/main/en/main_classes/configuration#transformers.PreTrainedConfig)) : Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [from_pretrained()](/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) method to load the model weights. This model inherits from [PreTrainedModel](/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel). Check the superclass documentation for the generic methods the library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads etc.) This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. #### forward[[transformers.InklingPreTrainedModel.forward]] ```python forward(*args, **kwargs) ``` A mock value for a dotted path (e.g. `torch.float32`): attribute access chains, calls behave as pass-through decorators, `repr` is the dotted path, and using it as a base class substitutes a plain-`type` base (PEP 560 `__mro_entries__`), so real subclasses keep a normal metaclass and `inspect.signature` reads their real `__init__` instead of a mock's. ## InklingTextModel[[transformers.InklingTextModel]] #### transformers.InklingTextModel[[transformers.InklingTextModel]] ```python transformers.InklingTextModel(config: InklingTextConfig) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/modeling_inkling.py#L646) **Parameters:** config ([InklingTextConfig](/docs/transformers/main/en/model_doc/inkling#transformers.InklingTextConfig)) : Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [from_pretrained()](/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) method to load the model weights. The bare Inkling Text Model outputting raw hidden-states without any specific head on top. This model inherits from [PreTrainedModel](/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel). Check the superclass documentation for the generic methods the library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads etc.) This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. #### forward[[transformers.InklingTextModel.forward]] ```python forward(input_ids: typing.Optional[torch.LongTensor] = None, attention_mask: typing.Optional[torch.Tensor] = None, position_ids: typing.Optional[torch.LongTensor] = None, past_key_values: transformers.cache_utils.Cache | None = None, inputs_embeds: typing.Optional[torch.FloatTensor] = None, use_cache: bool | None = None, **kwargs: Unpack) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/modeling_inkling.py#L665) **Parameters:** input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) : Indices of input sequence tokens in the vocabulary. Padding will be ignored by default. Indices can be obtained using [AutoTokenizer](/docs/transformers/main/en/model_doc/auto#transformers.AutoTokenizer). See [PreTrainedTokenizer.encode()](/docs/transformers/main/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.encode) and [PreTrainedTokenizer.__call__()](/docs/transformers/main/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.__call__) for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*) : Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) position_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) : Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. [What are position IDs?](../glossary#position-ids) past_key_values (`~cache_utils.Cache`, *optional*) : Pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used to speed up sequential decoding. This typically consists in the `past_key_values` returned by the model at a previous stage of decoding, when `use_cache=True` or `config.use_cache=True`. Only [Cache](/docs/transformers/main/en/internal/generation_utils#transformers.Cache) instance is allowed as input, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). If no `past_key_values` are passed, [DynamicCache](/docs/transformers/main/en/internal/generation_utils#transformers.DynamicCache) will be initialized by default. The model will output the same cache format that is fed as input. If `past_key_values` are used, the user is expected to input only unprocessed `input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, unprocessed_length)` instead of all `input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*) : Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. use_cache (`bool`, *optional*) : If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). **Returns:** [BaseModelOutputWithPast](/docs/transformers/main/en/main_classes/output#transformers.modeling_outputs.BaseModelOutputWithPast) or `tuple(torch.FloatTensor)` A [BaseModelOutputWithPast](/docs/transformers/main/en/main_classes/output#transformers.modeling_outputs.BaseModelOutputWithPast) or a tuple of `torch.FloatTensor` (if `return_dict=False` is passed or when `config.return_dict=False`) comprising various elements depending on the configuration ([InklingConfig](/docs/transformers/main/en/model_doc/inkling#transformers.InklingConfig)) and inputs. The [InklingTextModel](/docs/transformers/main/en/model_doc/inkling#transformers.InklingTextModel) forward method, overrides the `__call__` special method. Although the recipe for forward pass needs to be defined within this function, one should call the `Module` instance afterwards instead of this since the former takes care of running the pre and post processing steps while the latter silently ignores them. - **last_hidden_state** (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`) -- Sequence of hidden-states at the output of the last layer of the model. If `past_key_values` is used only the last hidden-state of the sequences of shape `(batch_size, 1, hidden_size)` is output. - **past_key_values** (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`) -- It is a [Cache](/docs/transformers/main/en/internal/generation_utils#transformers.Cache) instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). Contains pre-computed hidden-states (key and values in the self-attention blocks and optionally if `config.is_encoder_decoder=True` in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. - **hidden_states** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`) -- Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. - **attentions** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`) -- Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. ## InklingVisionModel[[transformers.InklingVisionModel]] #### transformers.InklingVisionModel[[transformers.InklingVisionModel]] ```python transformers.InklingVisionModel(config: InklingVisionConfig) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/modeling_inkling.py#L957) **Parameters:** config ([InklingVisionConfig](/docs/transformers/main/en/model_doc/inkling#transformers.InklingVisionConfig)) : Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [from_pretrained()](/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) method to load the model weights. The bare Inkling Model outputting raw hidden-states without any specific head on top. This model inherits from [PreTrainedModel](/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel). Check the superclass documentation for the generic methods the library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads etc.) This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. #### forward[[transformers.InklingVisionModel.forward]] ```python forward(pixel_values: Tensor, **kwargs: Unpack) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/modeling_inkling.py#L989) **Parameters:** pixel_values (`torch.Tensor` of shape `(batch_size, num_channels, image_size, image_size)`) : The tensors corresponding to the input images. Pixel values can be obtained using `image_processor_class`. See `image_processor_class.__call__` for details (`processor_class` uses `image_processor_class` for processing images). **Returns:** [BaseModelOutputWithPooling](/docs/transformers/main/en/main_classes/output#transformers.modeling_outputs.BaseModelOutputWithPooling) or `tuple(torch.FloatTensor)` A [BaseModelOutputWithPooling](/docs/transformers/main/en/main_classes/output#transformers.modeling_outputs.BaseModelOutputWithPooling) or a tuple of `torch.FloatTensor` (if `return_dict=False` is passed or when `config.return_dict=False`) comprising various elements depending on the configuration ([InklingConfig](/docs/transformers/main/en/model_doc/inkling#transformers.InklingConfig)) and inputs. The [InklingVisionModel](/docs/transformers/main/en/model_doc/inkling#transformers.InklingVisionModel) forward method, overrides the `__call__` special method. Although the recipe for forward pass needs to be defined within this function, one should call the `Module` instance afterwards instead of this since the former takes care of running the pre and post processing steps while the latter silently ignores them. - **last_hidden_state** (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`) -- Sequence of hidden-states at the output of the last layer of the model. - **pooler_output** (`torch.FloatTensor` of shape `(batch_size, hidden_size)`) -- Last layer hidden-state of the first token of the sequence (classification token) after further processing through the layers used for the auxiliary pretraining task. E.g. for BERT-family of models, this returns the classification token after processing through a linear layer and a tanh activation function. The linear layer weights are trained from the next sentence prediction (classification) objective during pretraining. - **hidden_states** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`) -- Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. - **attentions** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`) -- Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. ## InklingImageProcessor[[transformers.InklingImageProcessor]] #### transformers.InklingImageProcessor[[transformers.InklingImageProcessor]] ```python transformers.InklingImageProcessor(**kwargs: Unpack) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/image_processing_inkling.py#L60) **Parameters:** do_convert_rgb (`bool`, *kwargs*, *optional*, defaults to `True`) : Whether to convert the image to RGB. do_resize (`bool`, *kwargs*, *optional*) : Whether to resize the image. size (`Annotated[int | list[int] | tuple[int, ...] | dict[str, int] | None, None]`, *kwargs*, defaults to `{'height' : 40, 'width': 40}`): Describes the maximum input dimensions to the model. default_to_square (`bool`, *kwargs*, *optional*, defaults to `False`) : Whether to default to a square image when resizing, if size is an int. crop_size (`Annotated[int | list[int] | tuple[int, ...] | dict[str, int] | None, None]`, *kwargs*) : Size of the output image after applying `center_crop`. resample (`Annotated[Union[int, PILImageResampling, NoneType], None]`, *kwargs*, defaults to `Resampling.LANCZOS`) : Resampling filter to use if resizing the image. This can be one of the enum `PILImageResampling`. Only has an effect if `do_resize` is set to `True`. do_rescale (`bool`, *kwargs*, *optional*, defaults to `True`) : Whether to rescale the image. rescale_factor (`float`, *kwargs*, *optional*, defaults to `0.00392156862745098`) : Rescale factor to rescale the image by if `do_rescale` is set to `True`. do_normalize (`bool`, *kwargs*, *optional*, defaults to `True`) : Whether to normalize the image. image_mean (`Union[float, list[float], tuple[float, ...]]`, *kwargs*, *optional*, defaults to `[0.48145466, 0.4578275, 0.40821073]`) : Image mean to use for normalization. Only has an effect if `do_normalize` is set to `True`. image_std (`Union[float, list[float], tuple[float, ...]]`, *kwargs*, *optional*, defaults to `[0.26862954, 0.26130258, 0.27577711]`) : Image standard deviation to use for normalization. Only has an effect if `do_normalize` is set to `True`. do_pad (`bool`, *kwargs*, *optional*) : Whether to pad the image. Padding is done either to the largest size in the batch or to a fixed square size per image. The exact padding strategy depends on the model. pad_size (`Annotated[int | list[int] | tuple[int, ...] | dict[str, int] | None, None]`, *kwargs*) : The size in `{"height": int, "width" int}` to pad the images to. Must be larger than any image size provided for preprocessing. If `pad_size` is not provided, images will be padded to the largest height and width in the batch. Applied only when `do_pad=True.` do_center_crop (`bool`, *kwargs*, *optional*) : Whether to center crop the image. data_format (`Union[str, ~image_utils.ChannelDimension]`, *kwargs*, *optional*) : Only `ChannelDimension.FIRST` is supported. Added for compatibility with slow processors. input_data_format (`Union[str, ~image_utils.ChannelDimension]`, *kwargs*, *optional*) : The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. device (`Annotated[Union[str, torch.device, NoneType], None]`, *kwargs*) : The device to process the videos on. If unset, the device is inferred from the input videos. return_tensors (`Annotated[str | ~utils.generic.TensorType | None, None]`, *kwargs*) : Returns stacked tensors if set to `'pt'`, otherwise returns a list of tensors. disable_grouping (`bool`, *kwargs*, *optional*) : Whether to disable grouping of images by size to process them individually and not in batches. If None, will be set to True if the images are on CPU, and False otherwise. This choice is based on empirical observations, as detailed here: https://github.com/huggingface/transformers/pull/38157 image_seq_length (`int`, *kwargs*, *optional*) : The number of image tokens to be used for each image in the input. Added for backward compatibility but this should be set as a processor attribute in future models. rescale_image_frac (`float`, *kwargs*, *optional*) : Factor applied to the image's long edge before patch division, preserving aspect ratio. `None` disables the pre-scaling. rescale_image_max_upscaled_long_edge (`int`, *kwargs*, *optional*) : Cap, in pixels, on the upscaled long edge. Only limits growth: an image already above the cap is kept as is. Constructs a Inkling image processor. ## InklingFeatureExtractor[[transformers.InklingFeatureExtractor]] #### transformers.InklingFeatureExtractor[[transformers.InklingFeatureExtractor]] ```python transformers.InklingFeatureExtractor(feature_size: int = 80, sampling_rate: int = 16000, padding_value: float = 0.0, audio_token_duration_s: float = 0.05, window_size_multiplier: float = 2.0, n_fft: int | None = None, **kwargs) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/feature_extraction_inkling.py#L38) **Parameters:** feature_size (`int`, *optional*, defaults to 80) : The feature dimension of the extracted features, i.e. the number of mel filterbanks. sampling_rate (`int`, *optional*, defaults to 16000) : The sampling rate at which the audio files should be digitized, expressed in hertz (Hz). padding_value (`float`, *optional*, defaults to 0.0) : The value used to pad the log-mel spectrograms to the same length in a batch. audio_token_duration_s (`float`, *optional*, defaults to 0.05) : Duration, in seconds, represented by a single audio token, i.e. the STFT hop length. window_size_multiplier (`float`, *optional*, defaults to 2.0) : Multiplier applied to `audio_token_duration_s` to obtain the STFT window length. n_fft (`int`, *optional*) : FFT size. Defaults to the window length (`audio_token_duration_s * window_size_multiplier * sampling_rate`) when not provided. Constructs a TML audio feature extractor, which converts raw audio waveforms into log-mel spectrogram features (mel filterbank energies in log10 space). The quantization of these features into discrete dMel bins is performed downstream by [InklingProcessor](/docs/transformers/main/en/model_doc/inkling#transformers.InklingProcessor). This feature extractor inherits from [SequenceFeatureExtractor](/docs/transformers/main/en/main_classes/feature_extractor#transformers.SequenceFeatureExtractor) which contains most of the main methods. Users should refer to this superclass for more information regarding those methods. ## InklingProcessor[[transformers.InklingProcessor]] #### transformers.InklingProcessor[[transformers.InklingProcessor]] ```python transformers.InklingProcessor(feature_extractor = None, image_processor = None, tokenizer = None, chat_template = None, image_token = '<|unused_200054|>', audio_token = '<|unused_200053|>', image_bos_token = '<|content_image|>', audio_bos_token = '<|content_audio_input|>', num_dmel_bins = 16, dmel_min_value = -7.0, dmel_max_value = 2.0, **kwargs) ``` [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/processing_inkling.py#L34) **Parameters:** feature_extractor (`feature_extractor_class`) : The feature extractor is a required input. image_processor (`image_processor_class`) : The image processor is a required input. tokenizer (`tokenizer_class`) : The tokenizer is a required input. chat_template (`str`) : A Jinja template to convert lists of messages in a chat into a tokenizable string. image_token (`str`, *optional*, defaults to `"<|unused_200054|>"`) : Placeholder token for each image soft-token slot (replaced by image features). audio_token (`str`, *optional*, defaults to `"<|unused_200053|>"`) : Placeholder token for each audio soft-token slot (replaced by audio features). image_bos_token (`str`, *optional*, defaults to `"<|content_image|>"`) : Marker token that begins an image span (kept as an ordinary embedded token). audio_bos_token (`str`, *optional*, defaults to `"<|content_audio_input|>"`) : Marker token that begins an audio span (kept as an ordinary embedded token). num_dmel_bins (`int`, *optional*, defaults to 16) : Number of discrete bins each (clamped) log-mel value is quantized into. dmel_min_value (`float`, *optional*, defaults to -7.0) : Lower clamp bound, in log10 space, used for dMel quantization. dmel_max_value (`float`, *optional*, defaults to 2.0) : Upper clamp bound, in log10 space, used for dMel quantization. Constructs a InklingProcessor which wraps a feature extractor, a image processor, and a tokenizer into a single processor. [InklingProcessor](/docs/transformers/main/en/model_doc/inkling#transformers.InklingProcessor) offers all the functionalities of `feature_extractor_class`, `image_processor_class`, and `tokenizer_class`. See the `~feature_extractor_class`, `~image_processor_class`, and `~tokenizer_class` for more information.