Getting Started with Azure AI Foundry in Python | Connect Deepseek, llama, Phi, OpenAI

Learn how to get started with Azure AI Foundry in Python. Connect Deepseek, Meta Llama, Phi, and OpenAI models seamlessly from Azure AI with this step-by-step guide.

Prerequisites

1. Please follow our previous video where we created the Azure AI Foundry Project and Hub.
2. Visual Studio Code (VS Code).
3. Python 3.1x.

Detailed Video Tutorial

Get Started with Azure AI Foundry using Python

Sign in with the Azure CLI

Sign in with the Azure CLI with the same credentials that you used to create the AI Foundry Project. Also, select the same subscription during sign in process.

Open any terminal or command prompt and write this command to sign in to Azure. If you do not have Azure CLI, go ahead and install it first –

az login

Create a new Python Virtual Environment in VS Code.

Open VS Code on your developer machine. Click on View on top and select Command Palette…

Search for Python and from the search results, select Python: Create Environment… You will only get this option if you python installed on your machine. If you still do not get this option, you can use your terminal to create a new environment and activate it. Follow this tutorial to know How to Create a New Python Environment and Activate it in Windows OS?.

Select the Environment type. I usually go with Venv. It will be your preference, what do you use.

Select a Python installation to create the virtual environment. I will go with the latest version of Python. At the time of writing this blog, the latest version is Python 3.13.2. Once selected, it will create a new virtual environment in your current workspace or directory.

This is how it looks like in the file explorer. A new folder with name .venv is created.

Open a new terminal in VS Code. If you see .venv written before the path, then it confirms that your python virtual environment is successfully created and it also activated.

Now, if you install any package/libraries, it will get installed in your virtual environment and will not affect or create conflict in your other projects.

Create the Azure AI Project Client

Open a new terminal where you have virtual environment activated. Execute the below command to Install the Azure AI projects client library:

pip install azure-ai-projects azure-identity python-dotenv

Create a new python file main.py in your root directory of your project. Make sure, not inside the .venv folder. Also, create a new file .env in your root directory of your project.

We will use main.py to write our script to connect to AI models and .env will contain the secrets and environment variables. This is how your project structure will look like.

Create a project client in main.py.

from azure.identity import AzureCliCredential
from azure.ai.projects import AIProjectClient

## Get config from .env file
import os
from dotenv import load_dotenv
load_dotenv()

PROJECT_CONNECTION_STRING = os.getenv("PROJECT_CONNECTION_STRING")

# Create a client using the connection string
project = AIProjectClient.from_connection_string(PROJECT_CONNECTION_STRING, credential=AzureCliCredential())

Azure OpenAI Service

We have successfully created the AI Project Client. Let’s use it to connect to Azure OpenAI models. The Azure OpenAI Service provides access to OpenAI’s models including the GPT-4o, GPT-4o mini, GPT-4, GPT-4 Turbo with Vision, DALLE-3, Whisper, and Embeddings model series with the data residency, scalability, safety, security, and enterprise capabilities of Azure.

Let’s install the OpenAI SDK.

pip install openai

You can now use Project client to create Azure OpenAI client to make a call to Azure OpenAI models for text, image generation etc. Add the below code in your main.py file.

## Azure OpenAI Service
openai_client = project.inference.get_azure_openai_client(api_version="2024-06-01")
response_openai = openai_client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "What is the capital of France?"
        }
    ],
    temperature=0.7,
    max_tokens=50,
    top_p=1.0
)

print(response_openai.choices[0].message.content)

If you run the above code, you will end up getting an error as below.

openai.NotFoundError: Error code: 404 - {'error': {'code': 'DeploymentNotFound', 'message': 'The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.'}}

The reason being, we have not yet deployed any Azure OpenAI models in Azure AI Foundry portal.

Deploy AI Models in Azure AI Foundry

On the Azure AI Foundry portal, select your project and go to Model Catalog.

Select any of the chat completion models from Azure OpenAI. We have used gpt-4o.

Once you select any model, click on it. For the demonstration purpose, I am using gpt-4o-mini. Once, it shows the model details, click on Deploy.

Enter the deployment name and deployment type that best suits your AI solutions. I am going to keep the same name of model as deployment name. Also, I will select Global Standard as the type.

Click on Customize to customize your model for version, AI resource, capacity, etc.

Select necessary details and set the values suitable for your AI solutions. Finally, click on Deploy.

1. Enable automatic version updates: Enable this option to automatically update your model deployment within two weeks of a change to the default version. If you disable this option, you will need to manually update the model deployment version when the current version approaches retirement. Learn more .
2. Tokens per Minute Rate Limit: Assign TPM from your quota to set the rate limits for your deployment.
3. Content filter: Assign a content filter to a deployment to filter the content of the requests sent to the deployment.

All the above steps are exactly same for any model deployment not necessarily only from Azure OpenAI. You can deploy models from DeepSeek, Meta, Microsoft, Mistral, NVIDIA, etc.

Run the Project

In your terminal, execute the below command to run your python file:

python3 main.py

Once you run the project, you will get the similar output.

Azure AI model inference service

The Azure AI model inference service offers access to powerful models from leading providers like OpenAI, Microsoft, Meta, and more. These models support tasks such as content generation, summarization, and code generation.

To use the model inference service, first ensure that your project has an AI Services connection (in the management center).

Install the azure-ai-inference client library:

pip install azure-ai-inference

You can now use Project client to get a configured and authenticated Chat Completions Client or Embedding Client.

## Azure AI Inference
chat_inference = project.inference.get_chat_completions_client()
response_inference = chat_inference.complete(
    model="DeepSeek-V3",
    messages=[
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "What is the capital of India?"
        }
    ],
    temperature=0.7,
    max_tokens=50,
    top_p=1.0
)

print(response_inference.choices[0].message.content)

Replace the model with the deployment name. Here, I have used DeepSeek-V3. You can also use any other AI models as well such as Meta-Llama-3-8B-Instruct, Phi-4, Ministral-3B, etc.

Run the project and see the output on the terminal.

.venvjagdishkumawat@MacBookPro python % python3 main.py
The capital of France is **Paris**.
The capital of India is **New Delhi**. It is located in the northern part of the country and serves as the political and administrative center of India.

Source Code

from azure.identity import AzureCliCredential
from azure.ai.projects import AIProjectClient

## Get config from .env file
import os
from dotenv import load_dotenv
load_dotenv()

PROJECT_CONNECTION_STRING = os.getenv("PROJECT_CONNECTION_STRING")

# Create a client using the connection string
project = AIProjectClient.from_connection_string(PROJECT_CONNECTION_STRING, credential=AzureCliCredential())

## Azure OpenAI Service
openai_client = project.inference.get_azure_openai_client(api_version="2024-06-01")
response_openai = openai_client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "What is the capital of France?"
        }
    ],
    temperature=0.7,
    max_tokens=50,
    top_p=1.0
)

print(response_openai.choices[0].message.content)

## Azure AI Inference
chat_inference = project.inference.get_chat_completions_client()
response_inference = chat_inference.complete(
    model="DeepSeek-V3",
    messages=[
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "What is the capital of India?"
        }
    ],
    temperature=0.7,
    max_tokens=50,
    top_p=1.0
)

print(response_inference.choices[0].message.content)

Thank you All!!! Hope you find this useful.