156 lines
5.8 KiB
Markdown
156 lines
5.8 KiB
Markdown
# AI Services, File Storage, and PDF
|
|
|
|
> Part of `Nop.Plugin.Misc.FruitBankPlugin`. See `README.md` for project overview.
|
|
|
|
## FruitBankSettings
|
|
|
|
Plugin configuration stored in nopCommerce settings table (`ISettings`). Retrieved via `ISettingService`.
|
|
|
|
| Property | Type | Purpose |
|
|
|---|---|---|
|
|
| `ApiKey` | string | Cerebras API key |
|
|
| `ModelName` | string | Cerebras model name |
|
|
| `ApiBaseUrl` | string | Cerebras API base URL |
|
|
| `OpenAIApiKey` | string | OpenAI API key |
|
|
| `OpenAIModelName` | string | OpenAI model name |
|
|
| `OpenAIApiBaseUrl` | string | OpenAI API base URL |
|
|
| `MaxTokens` | int | Max response tokens |
|
|
| `Temperature` | double | Sampling temperature |
|
|
| `RequestTimeoutSeconds` | int | HTTP timeout |
|
|
| `IsEnabled` | bool | Plugin enabled flag |
|
|
|
|
Configured via admin UI: `FruitBankPluginAdminController` → Configure view.
|
|
|
|
## IAIAPIService Interface
|
|
|
|
Common contract for AI providers.
|
|
|
|
| Method | Signature | Purpose |
|
|
|---|---|---|
|
|
| `GetSimpleResponseAsync` | `(List<AIChatMessage>, CancellationToken?) -> Task<string>` | Non-streaming chat completion |
|
|
| `GetStreamedResponseAsync` | `(List<AIChatMessage>, CancellationToken?) -> IAsyncEnumerable<string>` | Streaming chat completion |
|
|
| `GetApiKey` | `() -> string` | Returns configured API key |
|
|
| `GetModelName` | `() -> string` | Returns configured model name |
|
|
|
|
**Chat message model:**
|
|
|
|
```
|
|
AIChatMessage { Role: "user"|"assistant"|"system", Content: string }
|
|
```
|
|
|
|
## OpenAIApiService
|
|
|
|
Primary AI provider. Implements `IAIAPIService` plus additional capabilities.
|
|
|
|
| Method | Purpose |
|
|
|---|---|
|
|
| `GetSimpleResponseAsync()` | Chat completion via OpenAI API |
|
|
| `GetStreamedResponseAsync()` | Streaming chat via SSE |
|
|
| `TranscribeAudioAsync(byte[] audioData, string language)` | Whisper API audio-to-text transcription |
|
|
| `GenerateImageAsync(string prompt)` | DALL-E image generation |
|
|
| `ExtractTextFromImageAsync(byte[] imageBytes)` | Vision API — extracts text from image |
|
|
| `AnalyzePdfWithAssistant(byte[] pdfBytes, string prompt)` | Vector store + Assistant API for PDF analysis |
|
|
| `UploadFileAsync(byte[], string purpose)` | File upload to OpenAI |
|
|
| `CreateVectorStoreAsync(string fileId)` | Creates vector store from uploaded file |
|
|
| `CreateThreadAndRunAsync(string vectorStoreId, string prompt)` | Runs assistant with vector store |
|
|
|
|
## CerebrasAPIService
|
|
|
|
Alternative fast-inference AI provider. Implements `IAIAPIService`.
|
|
|
|
| Method | Purpose |
|
|
|---|---|
|
|
| `GetSimpleResponseAsync()` | Chat completion via Cerebras API |
|
|
| `GetStreamedResponseAsync()` | Streaming chat via Cerebras API |
|
|
|
|
Tracks token usage via logging.
|
|
|
|
## ReplicateService
|
|
|
|
Image generation and processing via Replicate.com API.
|
|
|
|
| Method | Purpose |
|
|
|---|---|
|
|
| `GenerateImageAsync(string prompt)` | Image generation (Flux/Imagen model) |
|
|
| `GenerateLogoAsync(string prompt)` | Specialized logo generation |
|
|
| `RemoveBackgroundAsync(byte[] imageBytes)` | Background removal |
|
|
| `AnalyzeImageAsync(byte[] imageBytes, string prompt)` | Image analysis/description |
|
|
|
|
Uses `HttpClient` configured in `PluginNopStartup` with Replicate API token header.
|
|
|
|
## OpenAIService
|
|
|
|
Simple wrapper for quick single-prompt queries.
|
|
|
|
| Method | Signature | Purpose |
|
|
|---|---|---|
|
|
| `AskAsync` | `(string prompt) -> Task<string>` | Single prompt → response using GPT-4o-mini |
|
|
|
|
## AICalculationService
|
|
|
|
AI-powered business intelligence for the admin dashboard.
|
|
|
|
| Method | Purpose |
|
|
|---|---|
|
|
| `GetWelcomeMessageAsync()` | Generates personalized admin dashboard briefing |
|
|
|
|
**Aggregates:**
|
|
- Recent order data (count, totals, status distribution)
|
|
- Stock levels and discrepancies
|
|
- Weather information
|
|
- Flags inventory anomalies and processing issues
|
|
|
|
Called from `CustomDashboardController`.
|
|
|
|
## FileStorageService
|
|
|
|
File upload/download service with deduplication and compression.
|
|
|
|
| Method | Purpose |
|
|
|---|---|
|
|
| `SaveFileAsync(byte[], string fileName, ...)` | Save with hash calculation and optional compression |
|
|
| `GetFileAsync(int fileId)` | Retrieve file with automatic decompression if compressed |
|
|
| `SearchByFileNameAsync(string fileName)` | Search by original filename |
|
|
| `SearchByHashAsync(string hash)` | Find by SHA256 hash |
|
|
| `SearchByRawTextAsync(string text)` | Full-text search in extracted OCR content |
|
|
| `CalculateFileHashAsync(byte[])` | SHA256 hash calculation |
|
|
|
|
**Features:**
|
|
- **Deduplication**: Checks SHA256 hash before saving. Returns existing file if duplicate found
|
|
- **Compression**: Auto-compresses with GZip for non-compressed formats (excludes .jpg, .png, .gif, .zip, .gz, .pdf)
|
|
- **RawText**: Stores OCR-extracted text for full-text search
|
|
- **Metadata**: Tracks FileName, FileExtension, FileHash, FileSubPath, IsCompressed, Created/Modified
|
|
|
|
## IFileStorageProvider / LocalStorageProvider
|
|
|
|
Pluggable storage backend. Currently only `LocalStorageProvider` implemented.
|
|
|
|
| Method | Purpose |
|
|
|---|---|
|
|
| `SaveFileAsync(byte[], string relativePath)` | Save to `wwwroot/uploads/{relativePath}` |
|
|
| `GetFileAsync(string relativePath)` | Read file bytes |
|
|
| `DeleteFileAsync(string relativePath)` | Delete file, cleanup empty parent directories |
|
|
| `FileExistsAsync(string relativePath)` | Check file existence |
|
|
|
|
Files organized by relative sub-paths. Auto-creates directories as needed.
|
|
|
|
## PdfToImageService
|
|
|
|
PDF rendering using PDFtoImage library (Pdfium native backend).
|
|
|
|
| Method | Purpose |
|
|
|---|---|
|
|
| `ConvertPdfToJpgAsync(byte[] pdfBytes)` | Converts all PDF pages to JPG images |
|
|
|
|
- 300 DPI rendering, white background
|
|
- Returns `List<byte[]>` — one JPG per page
|
|
- Pdfium native library bootstrapped on first use
|
|
|
|
## Audio Transcription Flow
|
|
|
|
1. `FruitBankAudioController` receives audio recording
|
|
2. Calls `OpenAIApiService.TranscribeAudioAsync(audioData, language)`
|
|
3. Sends to Whisper API endpoint (`/v1/audio/transcriptions`)
|
|
4. Returns transcribed text
|
|
5. Used for voice-based order entry in FruitBankHybridApp
|