LLMOptions

Config file for the LLM model parameters.

Parameters:

Name	Type	Description	Default
temperature	float	Adjusts creativity in the model's responses (default: 0.8).	required
num_predict	int	Maximum tokens to predict in generation (default: -1).	required
mirostat	int	Enables Mirostat sampling for controlling perplexity (default: 0). - 0: Disabled - 1: Mirostat - 2: Mirostat 2.0	required
mirostat_eta	float	Adjusts the responsiveness of the Mirostat algorithm (default: 0.1).	required
mirostat_tau	float	Balances coherence and diversity in the output (default: 5.0).	required
num_ctx	int	Sets the size of the context window for token generation (default: 2048).	required
repeat_last_n	int	Specifies the range to prevent repetition (default: 64). - 0: Disabled - -1: Uses num_ctx	required
repeat_penalty	float	Penalizes repeated tokens in generation (default: 1.1).	required
temperature	float	Adjusts creativity in the model's responses (default: 0.8).	required
seed	int	Random seed for deterministic outputs (default: 0).	required
stop	str	Stop sequences to terminate generation (default: None).	required
tfs_z	float	Enables Tail Free Sampling to reduce less probable tokens (default: 1.0).	required
num_predict	int	Maximum tokens to predict in generation (default: -1).	required
top_k	int	Limits probability space for token selection (default: 40).	required
top_p	float	Controls cumulative probability for token selection (default: 0.9).	required
min_p	float	Ensures tokens meet a minimum probability threshold relative to the most likely token (default: 0.0).	required