Report / AI

Get data from OpenAI deployment/model

This sample shows how to use both OpenAI and document intelligene studio to process and extract data in a json format

Summary

This sample script integrates Azure's Document Intelligence and OpenAI services to extract from either images or pdfs or embedded images from any unstructured data and to use this script for any other purpose. Please change the system prompt in the script.

Functionality

  • Text Extraction: Utilizes Azure's Document Intelligence Read OCR model to extract text from documents, including PDFs and images.
  • Data Processing: Employs Azure OpenAI's GPT models to analyze the extracted text and identify key invoice details, such as Invoice Number, Sales Person, Date of Invoice, SubTotal, Tax, and Total.
  • JSON Output: Structures the identified information into a JSON format for streamlined data handling and integration.

Prerequisites

  • OpenAI API Key: Create an account here to obtain the API key.
  • Document Intelligence API Key: Create a key in the Document Intelligence Studio.
  • Settings: The settings of the script are part of set Settings-Agreements.json. Ensure that the script and its settings file are in the same location.

Settings

json
{
  "APIKeys": {
    "OpenAI": "<OCRKeyFromAzureOpenAI>",
    "DocumentIntelligence": "<OCRKeyFromADocumentIntelligence>"
  },
  "Endpoints": {
    "OpenAI": "https://<openaiendpoint>.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview",
    "DocumentIntelligence": "https://<docintelligendeendpoint>.cognitiveservices.azure.com"
  },
  "FilePath": "/Users/divyaakula/Documents/VidesoReview/SyntexDemo/AI Builder Document Processing Sample Data/Invoices/Adatum/Train/Adatum 1.pdf",
  "Base64String": null, 
  "OCRVersion": "2024-07-31-preview",
  "ChunkSize": 4194304
}

Example Screenshot

Example Screenshot


powershell
# A PowerShell script to extract invoice details using Azure OCR and OpenAI.

# Path to your config file
$configFilePath = "Settings-Agreements.json"
if (-not (Test-Path $configFilePath)) {
    throw "Configuration file not found at $configFilePath"
}

# Load configuration from JSON file
$Config = Get-Content -Path $configFilePath | ConvertFrom-Json

# Extract Values from Config
$OpenAI_Key = $Config.APIKeys.OpenAI
$OCR_Key = $Config.APIKeys.DocumentIntelligence
$OCR_Endpoint = $Config.Endpoints.DocumentIntelligence
$OpenAI_Endpoint = $Config.Endpoints.OpenAI
$OCR_Version = $Config.OCRVersion
$FilePath = $Config.FilePath
$Base64String = $Config.Base64String

# Function to Extract Text from PDF/Image Using Azure OCR
function Get-TextFromDocument($Base64String) {
    # Convert Base64 to binary
    $BinaryData = [Convert]::FromBase64String($Base64String)

    # Set OCR request headers
    $Headers = @{
        "Ocp-Apim-Subscription-Key" = $OCR_Key
        "Content-Type" = "application/pdf"  # Change to "image/jpeg" for images
    }

    # Send file to Azure Document Intelligence OCR API
    Invoke-RestMethod -Uri "$OCR_Endpoint/documentintelligence/documentModels/prebuilt-read:analyze?api-version=$OCR_Version" `
                      -Headers $Headers -Method Post -ResponseHeadersVariable HeaderInfo -Body $BinaryData

    if (-not $HeaderInfo) {
        throw "Failed to start OCR process."
    }

    # Extract operation URL
    $OperationUrl = ($HeaderInfo."Operation-Location")[0]
    
    # Poll for result
    Start-Sleep -Seconds 5  # Wait before checking status
    do {
        $Result = Invoke-RestMethod -Uri $OperationUrl -Headers @{ "Ocp-Apim-Subscription-Key" = $OCR_Key }
        $Status = $Result.status
        Start-Sleep -Seconds 2
    } while ($Status -ne "succeeded")

    # Extract recognized text
    $ExtractedText = ($Result.analyzeResult.pages.lines.content) -join " "
    return $ExtractedText
}

# Function to Extract Names Using Azure OpenAI
function Get-Inputs($ExtractedText) {
    # OpenAI Request Payload
    $OpenAI_Request = @{
        messages = @(
            @{ role = "system"; content = "Extract Invoice No, Sales Person, Date Of Invoice, SubTotal, Tax and Total and return in plain json format" }
            @{ role = "user"; content = "Text: $ExtractedText" }
        )
    } | ConvertTo-Json -Depth 3

    # OpenAI Headers
    $OpenAI_Headers = @{
        "api-key" = $OpenAI_Key
        "Content-Type" = "application/json"
    }

    # Call OpenAI API
    $OpenAI_Response = Invoke-RestMethod -Uri "$OpenAI_Endpoint" `
                                         -Method Post -Headers $OpenAI_Headers -Body $OpenAI_Request

    # Extract Names from Response
    $responseContent = $OpenAI_Response.choices[0].message.content 
    # Remove leading and trailing triple backticks and 'json' label
    $jsonString = $responseContent -replace '^```json\s*', '' -replace '\s*```$', ''
    return $jsonString
}

# Function to Process Base64 Document Input
function Get-Data($Base64String) {
    try {
        # Step 1: Extract text using OCR
        $ExtractedText = Get-TextFromDocument -Base64String $Base64String

        # Step 2: Extract Names using OpenAI
        $Names = Get-Inputs -ExtractedText $ExtractedText

        # Step 3: Return as JSON
        return $Names
    }
    catch {
        return @{ "error" = $_.Exception.Message } | ConvertTo-Json
    }
}

# Function to convert file to Base64 string
function Convert-Base64String($path) {
    $Base64Input = [Convert]::ToBase64String([IO.File]::ReadAllBytes($path))
    return $Base64Input
}


# Example Usage: Provide a Base64 String of a PDF or Image
$Base64Input = Convert-Base64String $FilePath

# Call the Function
$Result = Get-Data -Base64String $Base64Input
Write-Output $Result

More about OpenAI API


Contributors

Author(s) Author(s)
Divya Akula https://www.linkedin.com/in/ms-divya-akula/
Christian Veenhuis https://github.com/ChVeen

Disclaimer

THESE SAMPLES ARE PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING ANY IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR NON-INFRINGEMENT.

Tool documentation & references

  • Want to learn more about Azure OpenAI AI Deployments

    If you're interested in learning more about OpenAI API and its capabilities, this link provides a list of resources and tools to help you get started. Whether you're a developer looking to integrate Open AI into your applications or a researcher exploring the latest advances in artificial intelligence, this guide offers a range of helpful resources, including documentation, tutorials, and community forums.

  • Create and Configure an Azure OpenAI Resource

    Learn to create and configure an Azure OpenAI resource using the Azure portal for seamless AI integration.

Keep exploring