[NirajC-Microsoft]

Purpose

This pull request updates the data processing pipeline to use the Azure AI Foundry Project endpoint for generating embeddings, replacing the previous OpenAI endpoint and SDK integration. The changes span infrastructure outputs, environment variable handling, script interfaces, and the Python data processing logic, ensuring seamless integration with Azure AI Foundry and its EmbeddingsClient SDK.

Infrastructure and configuration updates:

• Added aiFoundryAiProjectEndpoint output to infra/modules/cognitive-services/main.bicep and exposed it in infra/main.bicep, making the AI Foundry Project endpoint available for downstream services. [1] [2]
• Updated infra/main.parameters.json to include the new aiFoundryAiProjectEndpoint parameter for deployment configuration.

Script and environment variable changes:

• Modified PowerShell (scripts/process_sample_data.ps1) and Bash (scripts/process_sample_data.sh) scripts to accept and export the new ProjectEndpoint parameter/environment variable, updating usage instructions in provisioning scripts accordingly. [1] [2] [3] [4] [5] [6]

Python data processing logic updates:

• Refactored scripts/index_scripts/02_process_data.py to use the Azure AI Inference SDK (azure-ai-inference) and the Foundry endpoint for embedding generation, replacing the OpenAI SDK and endpoint. [1] [2] [3]
• Updated Python requirements to use azure-ai-inference instead of openai.

Does this introduce a breaking change?

• Yes
• No

Golden Path Validation

• I have tested the primary workflows (the "golden path") to ensure they function correctly without errors.

Deployment Validation

• I have validated the deployment process successfully and all services are running as expected with this change.

What to Check

Verify that the following are valid

• ...

Other Information