API Reference

base_completion

BaseCompletion Objects

Python

class BaseCompletion()

Base class for handling completions. This class provides shared logic for creating completions,
both synchronously and asynchronously, and both streaming and non-streaming.

Attributes:

• endpoint str - API endpoint for the completion request.
• response_class Type - Class used for parsing the non-streaming response.
• stream_response_class Type - Class used for parsing the streaming response.

create

Python

@classmethod
def create(cls,
           model,
           prompt_or_messages=None,
           request_timeout=600,
           stream=False,
           **kwargs)

Create a completion or chat completion.

Arguments:

• model str - Model name to use for the completion.
• prompt_or_messages Union[str, List[ChatMessage]] - The prompt for Completion or a list of chat messages for ChatCompletion. If not specified, must specify either prompt or messages in kwargs.
• request_timeout int, optional - Request timeout in seconds. Defaults to 600.
• stream bool, optional - Whether to use streaming or not. Defaults to False.
• **kwargs - Additional keyword arguments.

Returns:

Union[CompletionResponse, Generator[CompletionStreamResponse, None, None]]:
Depending on the stream argument, either returns a CompletionResponse
or a generator yielding CompletionStreamResponse.

acreate

Python

@classmethod
def acreate(cls, model, *args, request_timeout=600, stream=False, **kwargs)

Asynchronously create a completion.

Arguments:

• model str - Model name to use for the completion.
• request_timeout int, optional - Request timeout in seconds. Defaults to 600.
• stream bool, optional - Whether to use streaming or not. Defaults to False.
• **kwargs - Additional keyword arguments.

Returns:

Union[CompletionResponse, AsyncGenerator[CompletionStreamResponse, None]]:
Depending on the stream argument, either returns a CompletionResponse or an async generator yielding CompletionStreamResponse.

completion

Completion Objects

Python

class Completion(BaseCompletion)

Class for handling text completions.

chat_completion

ChatCompletion Objects

Python

class ChatCompletion(BaseCompletion)

Class for handling chat completions.

api

Choice Objects

Python

class Choice(BaseModel)

A completion choice.

Attributes:

• index int - The index of the completion choice.
• text str - The completion response.
• logprobs float, optional - The log probabilities of the most likely tokens.
• finish_reason str - The reason the model stopped generating tokens. This will be "stop" if the model hit a natural stop point or a provided stop sequence, or "length" if the maximum number of tokens specified in the request was reached.

CompletionResponse Objects

Python

class CompletionResponse(BaseModel)

The response message from a /v1/completions call.

Attributes:

• id str - A unique identifier of the response.
• object str - The object type, which is always "text_completion".
• created int - The Unix time in seconds when the response was generated.
• choices List[Choice] - The list of generated completion choices.

CompletionResponseStreamChoice Objects

Python

class CompletionResponseStreamChoice(BaseModel)

A streamed completion choice.

Attributes:

• index int - The index of the completion choice.
• text str - The completion response.
• logprobs float, optional - The log probabilities of the most likely tokens.
• finish_reason str - The reason the model stopped generating tokens. This will be "stop" if the model hit a natural stop point or a provided stop sequence, or "length" if the maximum number of tokens specified in the request was reached.

CompletionStreamResponse Objects

Python

class CompletionStreamResponse(BaseModel)

The streamed response message from a /v1/completions call.

Attributes:

• id str - A unique identifier of the response.
• object str - The object type, which is always "text_completion".
• created int - The Unix time in seconds when the response was generated.
• model str - The model used for the chat completion.
  choices (List[CompletionResponseStreamChoice]):
  The list of streamed completion choices.

Model Objects

Python

class Model(BaseModel)

A model deployed to the Fireworks platform.

Attributes:

• id str - The model name.
• object str - The object type, which is always "model".
• created int - The Unix time in seconds when the model was generated.

ListModelsResponse Objects

Python

class ListModelsResponse(BaseModel)

The response message from a /v1/models call.

Attributes:

• object str - The object type, which is always "list".
• data List[Model] - The list of models.

ChatMessage Objects

Python

class ChatMessage(BaseModel)

A chat completion message.

Attributes:

• role str - The role of the author of this message.
• content str - The contents of the message.

ChatCompletionResponseChoice Objects

Python

class ChatCompletionResponseChoice(BaseModel)

A chat completion choice generated by a chat model.

Attributes:

• index int - The index of the chat completion choice.
• message ChatMessage - The chat completion message.
• finish_reason Optional[str] - The reason the model stopped generating tokens. This will be "stop" if the model hit a natural stop point or a provided stop sequence, or "length" if the maximum number of tokens specified in the request was reached.

UsageInfo Objects

Python

class UsageInfo(BaseModel)

Usage statistics.

Attributes:

• prompt_tokens int - The number of tokens in the prompt.
• total_tokens int - The total number of tokens used in the request (prompt + completion).
• completion_tokens Optional[int] - The number of tokens in the generated completion.

ChatCompletionResponse Objects

Python

class ChatCompletionResponse(BaseModel)

The response message from a /v1/chat/completions call.

Attributes:

• id str - A unique identifier of the response.
• object str - The object type, which is always "chat.completion".
• created int - The Unix time in seconds when the response was generated.
• model str - The model used for the chat completion.
• choices List[ChatCompletionResponseChoice] - The list of chat completion choices.
• usage UsageInfo - Usage statistics for the chat completion.

DeltaMessage Objects

Python

class DeltaMessage(BaseModel)

A message delta.

Attributes:

• role str - The role of the author of this message.
• content str - The contents of the chunk message.

ChatCompletionResponseStreamChoice Objects

Python

class ChatCompletionResponseStreamChoice(BaseModel)

A streamed chat completion choice.

Attributes:

• index int - The index of the chat completion choice.
• delta DeltaMessage - The message delta.
• finish_reason str - The reason the model stopped generating tokens. This will be "stop" if the model hit a natural stop point or a provided stop sequence, or "length" if the maximum number of tokens specified in the request was reached.

ChatCompletionStreamResponse Objects

Python

class ChatCompletionStreamResponse(BaseModel)

The streamed response message from a /v1/chat/completions call.

Attributes:

• id str - A unique identifier of the response.
• object str - The object type, which is always "chat.completion".
• created int - The Unix time in seconds when the response was generated.
• model str - The model used for the chat completion.
  choices (List[ChatCompletionResponseStreamChoice]):
  The list of streamed chat completion choices.

model

Model Objects

Python

class Model()

list

Python

@classmethod
def list(cls, request_timeout=60)

Returns a list of available models.

Arguments:

• request_timeout int, optional - The request timeout in seconds. Default is 60.

Returns:

• ListModelsResponse - A list of available models.

log

set_console_log_level

Python

def set_console_log_level(level: str) -> None

Controls console logging.

Arguments:

• level - the minimum level that prints out to console.
  Supported values: [CRITICAL, FATAL, ERROR, WARN,
  WARNING, INFO, DEBUG]

error

PermissionError Objects

Python

class PermissionError(FireworksError)

A permission denied error.

InvalidRequestError Objects

Python

class InvalidRequestError(FireworksError)

A invalid request error.

AuthenticationError Objects

Python

class AuthenticationError(FireworksError)

A authentication error.

RateLimitError Objects

Python

class RateLimitError(FireworksError)

A rate limit error.

InternalServerError Objects

Python

class InternalServerError(FireworksError)

An internal server error.

ServiceUnavailableError Objects

Python

class ServiceUnavailableError(FireworksError)

A service unavailable error.