From 7e964b07dc56d6917654ea5da3842cdd4fc8cd45 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 25 Feb 2025 09:45:03 +0000 Subject: [PATCH 01/68] update ApiClient doc --- docs/Api/ApiClient.md | 77 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 76 insertions(+), 1 deletion(-) diff --git a/docs/Api/ApiClient.md b/docs/Api/ApiClient.md index 1e6f7ae87..4633fd93c 100644 --- a/docs/Api/ApiClient.md +++ b/docs/Api/ApiClient.md @@ -22,7 +22,7 @@ When using HttpClient directly, you should manually take care of: * Circuit breaking (to avoid server overloading) ## Olive.ApiClient -Olive provides a helper utility class named [ApiClient](https://github.com/Geeksltd/Olive/tree/master/Integration/Olive.ApiClient) which handles all of the above issues for you. +Olive provides a helper utility class named [ApiClient](https://github.com/Geeksltd/Olive/tree/master/Olive.ApiClient) which handles all of the above issues for you. It's built on top of the standard HttpClient, and is ideal for invoking Web Apis. The following simple line is enough to take care of all aspects of downloading a customer object from a remote Web Api. @@ -216,3 +216,78 @@ If http exceptions are raised consecutively for the specified number of times, i During the break period, any attempt to execute a new request will **immediately throw a BrokenCircuitException**. Once the duration is over, if the first action throws http exception again, the circuit will break again for the same duration. Otherwise the circuit will reset. + +## HTTP Methods +### GET Request + +- **Purpose**: Retrieves data from the API. +- **Usage**: +```csharp +var data = await new ApiClient($"{baseUrl}/customers").Get(new { id = 123 }); +``` + +### POST Request + +- **Purpose**: Sends data to the server. +- **Usage**: +```csharp +var data = await new ApiClient($"{baseUrl}/customers").Post(new { name = CustomerName }); +``` + +### PUT Request + +- **Purpose**: Updates a resource on the server. +- **Usage**: +```csharp +var data = await new ApiClient($"{baseUrl}/customers").Put(new { name = CustomerName }); +``` + +### DELETE Request + +- **Purpose**: Deletes a resource. +- **Usage**: +```csharp +var data = await new ApiClient($"{baseUrl}/customers").Delete(new { id = 123 }); +``` + +### PATCH Request + +- **Purpose**: Partially updates a resource. +- **Usage**: +```csharp +var data = await new ApiClient($"{baseUrl}/customers").Patch(new { id = 123 }); +``` + +## Caching + +- **Purpose**: Enables caching for GET requests. +- **Usage**: +```csharp +var data = await new ApiClient($"{baseUrl}/customers") + .Cache(CachePolicy.FreshOrCacheOrFail, TimeSpan.FromMinutes(10)) + .Get(new { id = 123 }); +``` + +## Custom Headers + +- **Purpose**: Adds custom headers to API requests. +- **Usage**: +```csharp +var client = new ApiClient($"{baseUrl}/customers").Header(headers => headers.Add("Authorization", "Bearer token")); +``` + +## Authentication + +- **Purpose**: Authenticates the client using cookies. +- **Usage**: +```csharp +var client = new ApiClient($"{baseUrl}/customers").Authenticate(cookie1, cookie2); +``` + +## Deletes All Cache + +- **Purpose**: Deletes all cached Get API results. +- **Usage**: +```csharp +await ApiClient.DisposeCache(); +``` From 0eae8a64e1ab7033c773ffee46cdbf2658b275c0 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 25 Feb 2025 11:48:06 +0000 Subject: [PATCH 02/68] add Olive.Audit.DatabaseLogger doc --- docs/Logging/Olive.Audit.DatabaseLogger.md | 28 ++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 docs/Logging/Olive.Audit.DatabaseLogger.md diff --git a/docs/Logging/Olive.Audit.DatabaseLogger.md b/docs/Logging/Olive.Audit.DatabaseLogger.md new file mode 100644 index 000000000..f8414bc36 --- /dev/null +++ b/docs/Logging/Olive.Audit.DatabaseLogger.md @@ -0,0 +1,28 @@ +# Olive.Audit.DatabaseLogger + +## Overview +The `DatabaseLogger` class is responsible for logging audit events into a database. It implements the `IAuditLogger` interface and provides a mechanism to dynamically determine and instantiate audit event types. + +## Features +- Supports dynamic resolution of IAuditEvent implementations. +- Logs audit events asynchronously into the database. +- Provides an extension method for easy integration with `IServiceCollection`. + +## Extension Method: `DatabaseLoggerExtensions` +The `DatabaseLoggerExtensions` class provides a convenient method to register the `DatabaseLogger` in the service collection. + +### `AddDatabaseLogger(IServiceCollection services)` + +- **Purpose**: Registers `DatabaseLogger` as a transient service implementing `IAuditLogger`. +- **Usage**: +```csharp +public override void ConfigureServices(IServiceCollection services) +{ + ... + services.AddDatabaseLogger(); + ... +} +``` + +## Conclusion +The `DatabaseLogger` class provides a flexible and extensible solution for logging audit events into a database. Its dynamic event type resolution and seamless DI integration make it a powerful tool for auditing systems. \ No newline at end of file From ead1600d0deb088e099469f6d635a63bff1ff413 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 25 Feb 2025 13:53:29 +0000 Subject: [PATCH 03/68] update Olive.Audit doc --- docs/Logging/Audit.md | 172 ++++++++++++++++++++++++++++++++++-------- 1 file changed, 139 insertions(+), 33 deletions(-) diff --git a/docs/Logging/Audit.md b/docs/Logging/Audit.md index a6a598504..565870cb4 100644 --- a/docs/Logging/Audit.md +++ b/docs/Logging/Audit.md @@ -1,38 +1,105 @@ # Olive.Audit -*Olive.Audit* helps you to log and record application events such as **Insert**, **Update**, **Delete** or other application event. *Olive.Audit* saves the event records into database with **Olive.Audit.DatabaseLogger** provider. +**Olive.Audit** provides structured logging for entity changes, ensuring that all insertions, updates, and deletions are efficiently recorded. This includes entity processing, data extraction, and database storage. The event records are saved into the database using the **Olive.Audit.DatabaseLogger** provider. -## Logging events +## Installation Steps -You can log application events in *M# web apps* by using these methods: +### 1. Install Required Packages +Install the following NuGet packages: +- `Olive.Audit` +- `Olive.Audit.DatabaseLogger` + +### 2. Define the `AuditEvent` Entity +Add the `AuditEvent` entity to the **Domain**: + +```csharp +using MSharp; + +namespace Domain +{ + class AuditEvent : EntityType + { + public AuditEvent() + { + Implements("Olive.Audit.IAuditEvent"); + + DateTime("Date").Mandatory().DefaultToNow(); + String("Event").Mandatory(); + String("Item Id"); + String("Item Type"); + String("Item Group"); + String("User Id"); + String("User IP"); + BigString("Item Data"); + } + } +} +``` + +### 3. Implement `ContextUserProvider` +Create a `ContextUserProvider` class in **Domain** and implement `IContextUserProvider`: ```csharp -//Recording an update event -await Audit.LogUpdate(entity) +public class ContextUserProvider : IContextUserProvider +{ + public ClaimsPrincipal GetUser() + { + return Context.Current.User(); + } + public string GetUserIP() + { + return Context.Current.Http().Connection.RemoteIpAddress?.ToString(); + } +} +``` -//Recording a detele event -await Audit.LogDelete(entity) +### 4. Register Dependencies -//Recording an insert event -await Audit.LogInsert(entity) +Add the following services in `Startup.cs`: +```csharp +public override void ConfigureServices(IServiceCollection services) +{ + ... + services.AddScoped(); + services.AddDatabaseLogger(); + ... +} ``` ->**Note:** `entity` is an object of a class that impelimented `IEntity` +### 5. Enable Automatic Data Logging +To enable automatic logging of entity operations (insert, update, delete), modify the `appSettings.json` file as follows: + +```json +"Database": { + ... + "Audit": { + "Insert": { "Action": true, "Data": true }, + "Update": { "Action": true, "Data": true }, + "Delete": { "Action": true, "Data": true } + } +} +``` -Also you can record uncategorized and custom events: +### 6. Exclude or Include Specific Entities +- To **exclude** certain entities from being logged, set `EnableLogging(false)` in their M# definition. +- To **log only specific entities**, add the `[LogEvents(true)]` attribute to the entity’s partial class: ```csharp -await Audit.Log(@event, data) +using Olive.Entities; -//Also you can pass entity, userId and userip +namespace Domain +{ + [LogEvents(true)] + public partial class EntityName + { + } +} ``` -### Customize Audit Service -To do so, you can simply extend the `Olive.Audit.DefaultAudit` or -implement the `Olive.Audit.IAudit`. +### 7. Customize the Audit Service +Extend `Olive.Audit.DefaultAudit` or implement `Olive.Audit.IAudit` to customize audit behavior. Example: -For example, to use the simplest approach, in your Domain project, add the following class: ```csharp namespace Domain { @@ -45,8 +112,8 @@ namespace Domain } ``` -Then you can inject the service to service collection in the `Startup.cs` -like the following code. +Register this service in `Startup.cs` **before** calling `base.ConfigureServices`: + ```csharp public override void ConfigureServices(IServiceCollection services) { @@ -54,27 +121,66 @@ public override void ConfigureServices(IServiceCollection services) base.ConfigureServices(services); } ``` -> **Important** you should add it before calling the base `ConfigureServices`. -### Checkout recorded logs +## Viewing Recorded Logs + +The `Olive.Audit.DatabaseLogger` stores records in the database. To view the recorded logs: -**Olive.Audit.DatabaseLogger** records information in the database. If you want to see the recorded informatin, open your SSMS, go on your database tables and search for a table named **AuditEvents**. Query this table and see the recorded information. +1. Open **SQL Server Management Studio (SSMS)**. +2. Navigate to your database tables. +3. Look for the table named **AuditEvents**. +4. Run a query to retrieve logged events. + +> **Note:** If using a different RDBMS, check the `AuditEvents` table using the appropriate database management tools. ![image](https://user-images.githubusercontent.com/22152065/37540926-092b877c-296e-11e8-9ecf-944597be8300.png) ->**Note:** We assume that you use *MSSQL Server* as your primary database. if you use any other RDBMS, go ahead and use related database management tools. The information is recorded on `AuditEvents` table. +## Audit Log Rendering +`Olive.Audit` provides two extension methods (available from version **2.1.113** onwards) for rendering logs in HTML: -## Automatic Data Logs +- `NewChangesToHtml()`: Converts new changes (Insert/Update) into HTML. +- `OldChangesToHtml()`: Converts previous data (Delete/Update) into HTML. -Olive allows you to enable automatic log creation for all data operations. This is a turnkey solution to have all user actions (insert, edit, delete) on all entities logged for audit trail purposes. To enable this, in your appSettings file set the following: -```json - "Database": { - ... - "Audit": { - "Insert": { "Action": true, "Data": true }, - "Update": { "Action": true, "Data": true }, - "Delete": { "Action": true, "Data": true }, +### Styling Audit Logs +These SCSS classes enable customization of audit log records: + +```scss +.audit-log { + // Base styles + + &.audit-log-old { + // Styles for old changes + } + + &.audit-log-new { + // Styles for new changes + } + + .audit-log-label { + // Styles for labels + } + + .audit-log-property { + // Styles for property values + + &.audit-log-property-bool { + // Styles for boolean properties + } + + &.audit-log-property-entity { + // Styles for associated entity values } - }, + } +} ``` + +## Summary + +**Olive.Audit** efficiently tracks entity changes, ensuring data integrity and accountability. It provides: +- Structured logging of entity modifications. +- Seamless database integration. +- Flexible configuration via `appSettings.json`. +- Easy customization via dependency injection. + +This makes it a robust solution for audit logging in M# web applications. \ No newline at end of file From b8e491d58bad06a5a3eff6127ae3573e221e6247 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 25 Feb 2025 15:05:25 +0000 Subject: [PATCH 04/68] add Olive.Aws.Comprehend doc --- docs/Services/Olive.Aws.Comprehend.md | 111 ++++++++++++++++++++++++++ 1 file changed, 111 insertions(+) create mode 100644 docs/Services/Olive.Aws.Comprehend.md diff --git a/docs/Services/Olive.Aws.Comprehend.md b/docs/Services/Olive.Aws.Comprehend.md new file mode 100644 index 000000000..9b00d81f7 --- /dev/null +++ b/docs/Services/Olive.Aws.Comprehend.md @@ -0,0 +1,111 @@ +# Olive.Aws.Comprehend + +## Overview +The `ComprehendClassifier` class provides functionality to interact with AWS Comprehend and AWS S3 for text classification. It supports creating and managing classifiers, running classification jobs, and handling data storage in S3. + +## Configuration +- **AWS:Comprehend:S3Bucket** - The default S3 bucket for storing classification data. +- **AWS:Comprehend:S3ClassificationJobsBucket** - The bucket for classification jobs. +- **AWS:Comprehend:Region** - The AWS region where Comprehend services are deployed. +- **AWS:Comprehend:IAMRoleArn** - IAM Role ARN required for Comprehend operations. + +## AWS Clients +```csharp +public static IAmazonS3 S3Client +public static IAmazonComprehend Client +``` +- **S3Client**: Provides access to AWS S3 operations. +- **Client**: Provides access to AWS Comprehend operations. + +### `Region(Amazon.RegionEndpoint region)` +```csharp +public static void Region(Amazon.RegionEndpoint region) +``` +- **Summary**: Creates and uses a new AWS client in the specified region. +- **Usage**: +```csharp +ComprehendClassifier.Region(RegionEndpoint.USEast1); +``` + +## Classifier Management + +### `CreateAsync` +```csharp +public static async Task CreateAsync(string name, string Version, LanguageCode languageCode = null, DocumentClassifierMode mode = null) +``` +- **Summary**: Creates and starts the training of Comprehend classification module based on AWS:Comprehend:S3Bucket bucket. +- **Returns**: The Amazon arn of the module needed to check the status and use the module.If Bucket is not specified will use Blob:S3:Bucket defualt bucket of the application. + +### `CreateAsync` +```csharp +public static async Task CreateAsync(string name, string Version, string bucketName, LanguageCode languageCode= null, DocumentClassifierMode mode = null) +``` +- **Summary**: Creates and starts the training of Comprehend classification module based on the bucket specified. +- **Returns**: The Amazon arn of the module needed to check the status and use the module. + +### `DescribeClasifier` +```csharp +public static async Task DescribeClasifier(string classifier_arn) +``` +- **Summary**: Retrieves full details of a classifier. +- **Returns**: `DocumentClassifierProperties` containing classifier details. + +### `GetClasifierStatus` +```csharp +public static async Task GetClasifierStatus(string classifier_arn) +``` +- **Summary**: Returns the current status of a classifier. + +### `DeleteClassifier` +```csharp +public static async Task DeleteClassifier(string classifier_arn) +``` +- **Summary**: Deletes a document classifier. + +## Classification Operations + +### `ClassifyByEndpoint` +```csharp +public static async Task ClassifyByEndpoint(string endpoint_arn, string documentText) +``` +- **Summary**: Classifies a document using a trained classifier endpoint. +- **Note**: Classifying by endpoint is costly; use `StartClasifyingJob` when possible. + +### `StartClasifyingJob` +```csharp +public static async Task StartClasifyingJob(string classifier_arn, string documetnKey, string outputfolder, string inputfolder) +``` +- **Summary**: Starts the Clasifying job based on the location in the s3 bucket configured AWS:Comprehend:S3ClassificationJobsBucket. +- **Returns**: Gives back the job id that you can use with GetClasifyingJobResults() to get the results of a job. + +### `GetClasifyingJobResults` +```csharp +public static async Task GetClasifyingJobResults(string jobId, string nextToken = null) +``` +- **Summary**: Returns an object containing the status of the classification job and the URI of the output file. + +## S3 File Management + +### `CopyFile` +```csharp +public static async Task CopyFile(string sourceKey, string Destinationkey) +``` +- **Summary**: Copies the file from AWS:Rekognition:S3SourceBucket to AWS:Comprehend:S3Bucket bucket that is the main comprehend bucket. +- **Returns**: The Checksum SHA256 of the saved file for confirmation if needed.If Bucket is not specified will use Blob:S3:Bucket defualt bucket of the application. + +### `CopyFile` +```csharp +public static async Task CopyFile(string sourceKey, string Destinationkey, string sourceBucket, string destinationBucket) +``` +- **Summary**: Copies the file from specified source bucket and destination bucket to copy a file. +- **Returns**: The Checksum SHA256 of the saved file for confirmation if needed. + +## Usage Example +```csharp +var classifierArn = await ComprehendClassifier.CreateAsync("MyClassifier", "v1"); +var status = await ComprehendClassifier.GetClasifierStatus(classifierArn); +await ComprehendClassifier.DeleteClassifier(classifierArn); +``` + +## Conclusion +The `ComprehendClassifier` class provides an easy-to-use wrapper for AWS Comprehend classification tasks, enabling efficient training, management, and execution of classification models. From 2fa2c31beb7e4cea8c7b150a6c909d4d19b1688d Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Wed, 26 Feb 2025 07:57:21 +0000 Subject: [PATCH 05/68] add Olive.Aws.EventBus doc --- docs/Services/Olive.Aws.EventBus.md | 187 ++++++++++++++++++++++++++++ 1 file changed, 187 insertions(+) create mode 100644 docs/Services/Olive.Aws.EventBus.md diff --git a/docs/Services/Olive.Aws.EventBus.md b/docs/Services/Olive.Aws.EventBus.md new file mode 100644 index 000000000..f858912cc --- /dev/null +++ b/docs/Services/Olive.Aws.EventBus.md @@ -0,0 +1,187 @@ +# Olive.Aws.EventBus + +## Overview +The `EventBusQueue` class provides a structured approach to interacting with Amazon SQS for event-driven message handling. It supports publishing messages, subscribing to events, and managing batch operations. + +## Configuration +To use this package, ensure the following configurations are set in your appsettings.json: +- **Aws:EventBusQueue:MaxNumberOfMessages** - The maximum number of messages retrieved in one batch (default: 10). +- **Aws:EventBusQueue:VisibilityTimeout** - The duration (in seconds) messages remain hidden after retrieval (default: 300). + +### Installation +To integrate this package into your project, install the necessary dependencies: +```sh +Install-Package AWSSDK.SQS +Install-Package Newtonsoft.Json +``` + +## AWS Clients +```csharp +public IAmazonSQS Client +``` +- **Client**: Provides access to AWS SQS operations. + +### `Region(Amazon.RegionEndpoint region)` +```csharp +public EventBusQueue Region(Amazon.RegionEndpoint region) +``` +- **Summary**: Creates and uses a new AWS SQS client for the specified region. +- **Usage**: +```csharp +var queue = new EventBusQueue("queue-url").Region(RegionEndpoint.USEast1); +``` + +## Queue Management + +### `Publish` +```csharp +public async Task Publish(string message) +``` +- **Summary**: Publishes a message to the queue. +- **Usage**: +```csharp +var queue = new EventBusQueue("queue-url"); +string messageId = await queue.Publish("{ \"message\": \"Hello World\" }"); +``` +- **Notes**: + - If the queue is FIFO and MessageDeduplicationId or MessageGroupId are empty, the method assigns `MessageDeduplicationId` and `MessageGroupId` automatically. + +### `PublishBatch` +```csharp +public async Task> PublishBatch(IEnumerable messages) +``` +- **Summary**: Publishes a batch of messages to the queue. +- **Usage**: +```csharp +var messages = new List { "message1", "message2" }; +var messageIds = await queue.PublishBatch(messages); +``` +- **Warning**: If too many messages fail, the method retries up to `MAX_RETRY(4)` times. + +### `Subscribe` +```csharp +public void Subscribe(Func handler) +``` +- **Summary**: Subscribes to the queue and processes incoming messages using the provided handler. +- **Usage**: +```csharp +queue.Subscribe(async message => +{ + Console.WriteLine($"Received: {message}"); +}); +``` + +### `PullAll` +```csharp +public async Task PullAll(Func handler) +``` +- **Summary**: Continuously pulls messages from the queue and processes them. +- **Usage**: +```csharp +await queue.PullAll(async message => +{ + Console.WriteLine($"Processing: {message}"); +}); +``` + +### `PullBatch` +```csharp +public async Task> PullBatch(int timeoutSeconds = 10, int? maxNumerOfMessages = null) +``` +- **Summary**: Retrieves a batch of messages from the queue. +- **Usage**: +```csharp +var messages = await queue.PullBatch(maxNumerOfMessages : 5); +foreach (var message in messages) +{ + Console.WriteLine(message.RawMessage); + await message.Complete(); +} +``` + +### `Purge` +```csharp +public Task Purge() +``` +- **Summary**: Clears all messages from the queue. +- **Warning**: This action is irreversible. +- **Usage**: +```csharp +await queue.Purge(); +``` + +## Event Bus Provider +The `EventBusProvider` class manages queue instances and caching. + +### `Provide` +```csharp +public IEventBusQueue Provide(string queueUrl) +``` +- **Summary**: Retrieves an `EventBusQueue` instance for the given queue URL. +- **Usage**: +```csharp +var provider = new EventBusProvider(); +var queue = provider.Provide("queue-url"); +``` + +## AWS Event Bus Extensions + +### `AddAwsEventBus` +```csharp +public static IServiceCollection AddAwsEventBus(this IServiceCollection @this) +``` +- **Summary**: Registers the AWS event bus service with the DI container. +- **Usage**: +```csharp +services.AddAwsEventBus(); +``` + +## SNS Message Handling +The `SnsMessage` and `SnsMessage` classes handle AWS SNS message processing. + +### `ParseMessage` +```csharp +public T ParseMessage() +``` +- **Summary**: Deserializes the SNS message body into the specified type. +- **Usage**: +```csharp +var snsMessage = new SnsMessage { Message = "{ \"name\": \"John\" }" }; +var parsedObject = snsMessage.ParseMessage(); +``` + +## Subscriber Class +The `Subscriber` class manages message polling from the queue. + +### `Start` +```csharp +public void Start() +``` +- **Summary**: Starts polling the queue for messages. + +### `PullAll` +```csharp +public Task PullAll() +``` +- **Summary**: Fetches and processes all available messages. + +## Full Example +```csharp +public override void ConfigureServices(IServiceCollection services) +{ + ... + services.AddAwsEventBus(); + ... +} +``` + +```csharp +var queueProvider = Context.Current.GetService(); +var queue = queueProvider.Provide("queue-url"); + +await queue.Publish("Hello, World!"); +queue.Subscribe(async message => Console.WriteLine($"Received: {message}")); +``` + +## Conclusion +The `EventBusQueue` system simplifies AWS SQS operations, providing an efficient mechanism for publishing, consuming, and managing messages. It includes error handling, retries, and supports FIFO queues seamlessly. From 44333cb32c613f1fbbbfa761b0d24932b13e3809 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Wed, 26 Feb 2025 08:54:57 +0000 Subject: [PATCH 06/68] Olive.Aws.LambdaFunction doc --- docs/Services/Olive.Aws.LambdaFunction.md | 162 ++++++++++++++++++++++ 1 file changed, 162 insertions(+) create mode 100644 docs/Services/Olive.Aws.LambdaFunction.md diff --git a/docs/Services/Olive.Aws.LambdaFunction.md b/docs/Services/Olive.Aws.LambdaFunction.md new file mode 100644 index 000000000..bad315e07 --- /dev/null +++ b/docs/Services/Olive.Aws.LambdaFunction.md @@ -0,0 +1,162 @@ +# Olive.Aws.LambdaFunction + +## Overview +The `Startup` and `Function` classes provide a structured way to configure and run AWS Lambda functions using .NET and Microsoft.Extensions.Hosting. These classes handle configuration, dependency injection, logging, and environment setup. + +## Configuration +Ensure the following configurations are set in your application: +- **Aws:ServiceUrl** - The base service URL for AWS services. +- **Logging** - Logging configuration section for different environments. + +## Installation +To integrate this package into your project, install the necessary dependencies: +```sh +Install-Package Microsoft.Extensions.Hosting +Install-Package Amazon.Lambda.Core +Install-Package Microsoft.Extensions.Logging +``` + +## Class: `Startup` + +### `ConfigureConfiguration` +```csharp +public virtual void ConfigureConfiguration(HostBuilderContext context, IConfigurationBuilder builder) +``` +- **Summary**: Configures the application configuration using `HostBuilderContext`. +- **Usage**: +```csharp +public override void ConfigureConfiguration(HostBuilderContext context, IConfigurationBuilder builder) +{ + base.ConfigureConfiguration(context, builder); +} +``` + +### `ConfigureServices` +```csharp +public virtual void ConfigureServices(IServiceCollection services) +``` +- **Summary**: Configures dependency injection and merges environment variables into the configuration. +- **Usage**: +```csharp +public override void ConfigureServices(IServiceCollection services) +{ + base.ConfigureServices(services); + services.AddSingleton(); +} +``` + +### `ConfigureLogging` +```csharp +public virtual void ConfigureLogging(HostBuilderContext context, ILoggingBuilder builder) +``` +- **Summary**: Configures logging based on the environment. +- **Notes**: + - In **development mode**, logs are sent to the console. + - In **production**, AWS logging provider is used. +- **Usage**: +```csharp +public override void ConfigureLogging(HostBuilderContext context, ILoggingBuilder builder) +{ + base.ConfigureLogging(context, builder); + builder.AddConsole(); +} +``` + +### `ConfigureHost` +```csharp +public virtual void ConfigureHost(IHostBuilder builder) +``` +- **Summary**: Custom host configurations can be added in derived classes. +- **Usage**: +```csharp +public override void ConfigureHost(IHostBuilder builder) +{ + builder.ConfigureServices(services => + { + services.AddSingleton(); + }); +} +``` + +## Class: `Function` + +### `Function Constructor` +```csharp +protected Function() : this(null) +protected Function(string[] args) +``` +- **Summary**: Initializes a new instance of the function and configures the host. + +### `Init` +```csharp +protected virtual void Init(IHostBuilder builder) +``` +- **Summary**: Sets up configuration, services, and logging. +- **Usage**: +```csharp +protected override void Init(IHostBuilder builder) +{ + base.Init(builder); + builder.ConfigureServices(services => + { + services.AddSingleton(); + }); +} +``` + +### `HostCreated` +```csharp +protected virtual void HostCreated(IHost host) +``` +- **Summary**: Called after the host is created. Initializes context and logging. +- **Usage**: +```csharp +protected override void HostCreated(IHost host) +{ + base.HostCreated(host); + Console.WriteLine("Lambda function initialized."); +} +``` + +### `LocalExecute` +```csharp +protected static Task LocalExecute(string[] args) + where TFunction : Function, new() +``` +- **Summary**: Allows local execution of the Lambda function. + +### `ExecuteAsync` +```csharp +public abstract Task ExecuteAsync(ILambdaContext context) +``` +- **Summary**: Abstract method that must be implemented in derived classes to handle AWS Lambda execution. +- **Usage**: +```csharp +public override async Task ExecuteAsync(ILambdaContext context) +{ + Console.WriteLine("Lambda function executed."); +} +``` + +## Usage Example +```csharp +public class MyLambdaStartup : Startup +{ + public override void ConfigureServices(IServiceCollection services) + { + base.ConfigureServices(services); + //services.AddSingleton(); + } +} + +public class MyLambdaFunction : Function +{ + public override async Task ExecuteAsync(ILambdaContext context) + { + Console.WriteLine("Executing Lambda Function"); + } +} +``` + +## Conclusion +The `Startup` and `Function` classes provide a robust foundation for building AWS Lambda functions with dependency injection, configuration management, and logging. These classes ensure a scalable and maintainable architecture. From f21f8be388f4d221fc2221371fcd569f0e4c1cf3 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Wed, 26 Feb 2025 10:03:32 +0000 Subject: [PATCH 07/68] Olive.Aws.Mvc doc --- docs/Services/Olive.Aws.Mvc.md | 150 +++++++++++++++++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 docs/Services/Olive.Aws.Mvc.md diff --git a/docs/Services/Olive.Aws.Mvc.md b/docs/Services/Olive.Aws.Mvc.md new file mode 100644 index 000000000..bd8b8aab5 --- /dev/null +++ b/docs/Services/Olive.Aws.Mvc.md @@ -0,0 +1,150 @@ +# Olive.Aws.Mvc + +## Overview +The `App` and `Startup` classes provide a structured way to configure and run AWS Lambda functions using ASP.NET Core. These classes handle configuration, dependency injection, logging, and request culture management. + +## Configuration +Ensure the following configurations are set in your application: +- **Logging** - Configuration section for logging setup. +- **Aws:ServiceUrl** - The base service URL for AWS services. +- **Aws:Secrets:Id** - The AWS Secrets Manager ID for data protection. + +## Installation +To integrate this package into your project, install the necessary dependencies: +```sh +Install-Package Microsoft.AspNetCore.Hosting +Install-Package Amazon.Lambda.AspNetCoreServer +Install-Package Amazon.Lambda.Logging.AspNetCore +``` + +## Class: `App` + +### `Init` +```csharp +protected override void Init(IWebHostBuilder builder) +``` +- **Summary**: Initializes the web host builder with the specified `TStartup` and configures logging. +- **Usage**: +```csharp +protected override void Init(IWebHostBuilder builder) +{ + base.Init(builder); + builder.ConfigureLogging(logging => logging.AddConsole()); +} +``` + +### `LocalRun(string[] args)` +```csharp +protected static void LocalRun(string[] args) where TApp : App, new() +``` +- **Summary**: Runs the application locally for testing purposes. + +### `LocalRun(TApp app, string[] args)` +```csharp +protected static void LocalRun(TApp app, string[] args) where TApp : App +``` +- **Summary**: Runs the specified instance of the application locally. + +### `ConfigureLogging` +```csharp +protected virtual void ConfigureLogging(WebHostBuilderContext context, ILoggingBuilder logging) +``` +- **Summary**: Configures logging for the application. +- **Notes**: + - In **development mode**, logs are sent to the console. + - In **production**, AWS Lambda logging is enabled with additional details. +- **Usage**: +```csharp +protected override void ConfigureLogging(WebHostBuilderContext context, ILoggingBuilder logging) +{ + base.ConfigureLogging(context, logging); +} +``` + +## Class: `Startup` + +### Constructor +```csharp +protected Startup(IWebHostEnvironment env, IConfiguration config) +``` +- **Summary**: Initializes the startup class and sets the default culture. +- **Usage**: +```csharp +public MyStartup(IWebHostEnvironment env, IConfiguration config) : base(env, config) { } +``` + +### `ConfigureServices` +```csharp +public override void ConfigureServices(IServiceCollection services) +``` +- **Summary**: Configures dependency injection and enables CORS. +- **Notes**: + - Enables CORS policy named `AllowOrigin`. + - If running in **production**, configures AWS data protection services. +- **Usage**: +```csharp +public override void ConfigureServices(IServiceCollection services) +{ + base.ConfigureServices(services); +} +``` + +### `ConfigureMvc` +```csharp +protected override void ConfigureMvc(IMvcBuilder mvc) +``` +- **Summary**: Configures MVC and Razor Pages options. +- **Usage**: +```csharp +protected override void ConfigureMvc(IMvcBuilder mvc) +{ + base.ConfigureMvc(mvc); +} +``` + +### `ConfigureRazorPagesOptions` +```csharp +protected virtual void ConfigureRazorPagesOptions(Microsoft.AspNetCore.Mvc.RazorPages.RazorPagesOptions options) +``` +- **Summary**: Allows customization of Razor Pages settings. +- **Usage**: +```csharp +protected override void ConfigureRazorPagesOptions(RazorPagesOptions options) +{ + +} +``` + +### `GetRequestCulture` +```csharp +protected override CultureInfo GetRequestCulture() +``` +- **Summary**: Sets the default request culture to `en-GB`. +- **Usage**: +```csharp +protected override CultureInfo GetRequestCulture() +{ + return new CultureInfo("fr-FR"); +} +``` + +## Full Example +```csharp +public class MyStartup : Startup +{ + public MyStartup(IWebHostEnvironment env, IConfiguration config) : base(env, config) { } + + public override void ConfigureServices(IServiceCollection services) + { + base.ConfigureServices(services); + //services.AddSingleton(); + } +} + +public class MyLambdaApp : App +{ +} +``` + +## Conclusion +The `App` and `Startup` classes provide a robust framework for building AWS Lambda applications using ASP.NET Core. These classes ensure proper logging, dependency injection, request culture handling, and allow local execution for easy testing and debugging. From d191701e80dbde97426bef2559a7e23f92506ca1 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Wed, 26 Feb 2025 10:29:22 +0000 Subject: [PATCH 08/68] Olive.Aws.Rekognition doc --- docs/Services/Olive.Aws.Rekognition.md | 98 ++++++++++++++++++++++++++ 1 file changed, 98 insertions(+) create mode 100644 docs/Services/Olive.Aws.Rekognition.md diff --git a/docs/Services/Olive.Aws.Rekognition.md b/docs/Services/Olive.Aws.Rekognition.md new file mode 100644 index 000000000..b512c559a --- /dev/null +++ b/docs/Services/Olive.Aws.Rekognition.md @@ -0,0 +1,98 @@ +# Olive.Aws.Rekognition + +## Overview +The `TextDetector` class provides functionality to interact with AWS Rekognition for text detection in images. It supports text extraction from images stored in S3 as well as images provided as byte streams. + +## Configuration +Ensure the following configurations are set in your application: +- **AWS:Rekognition:S3Bucket** - The default S3 bucket where images are stored. +- **AWS:Rekognition:Region** - The AWS region where Rekognition services are deployed. +- **Blob:S3:Bucket** - An alternative bucket for storing images. + +## Installation +To integrate this package into your project, install the necessary dependencies: +```sh +Install-Package AWSSDK.Rekognition +``` + +## AWS Clients +```csharp +public static IAmazonRekognition Client +``` +- **Client**: Provides access to AWS Rekognition operations. + +### `Region(Amazon.RegionEndpoint region)` +```csharp +public static void Region(Amazon.RegionEndpoint region) +``` +- **Summary**: Creates and uses a new AWS Rekognition client for the specified region. +- **Usage**: +```csharp +TextDetector.Region(RegionEndpoint.USEast1); +``` + +## Text Detection Operations + +### `GetTextDetetionResult` +```csharp +public static async Task> GetTextDetetionResult(string photo) +``` +- **Summary**: Retrieves text detection results for an image stored in the configured S3 bucket. +- **Usage**: +```csharp +var results = await TextDetector.GetTextDetetionResult("image.jpg"); +foreach (var text in results) +{ + Console.WriteLine(text.DetectedText); +} +``` +- **Warning**: Throws an exception if `AWS:Rekognition:S3Bucket` is not configured. + +### `GetTextDetetionResult` +```csharp +public static async Task> GetTextDetetionResult(string photo, string bucketName) +``` +- **Summary**: Retrieves text detection results for an image stored in a specific S3 bucket. +- **Usage**: +```csharp +var results = await TextDetector.GetTextDetetionResult("image.jpg", "custom-bucket"); +``` + +### `GetTextDetetionResult` +```csharp +public static async Task> GetTextDetetionResult(MemoryStream photo) +``` +- **Summary**: Retrieves text detection results for an image provided as a byte stream. +- **Usage**: +```csharp +using (var memoryStream = new MemoryStream(File.ReadAllBytes("image.jpg"))) +{ + var results = await TextDetector.GetTextDetetionResult(memoryStream); +} +``` + +## Error Handling +If an error occurs while processing the image, an exception is thrown. Ensure to wrap calls in a try-catch block: +```csharp +try +{ + var results = await TextDetector.GetTextDetetionResult("image.jpg"); +} +catch (Exception ex) +{ + Console.WriteLine($"Error: {ex.Message}"); +} +``` + +## Full Example +```csharp +TextDetector.Region(RegionEndpoint.USEast1); +var results = await TextDetector.GetTextDetetionResult("image.jpg"); +foreach (var text in results) +{ + Console.WriteLine(text.DetectedText); +} +``` + +## Conclusion +The `TextDetector` class provides an easy-to-use wrapper for AWS Rekognition text detection, enabling efficient extraction of text from images stored in S3 or processed as byte streams. \ No newline at end of file From 4d30e93f807d3c700497dfdfd65ce17bc6ff1061 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Wed, 26 Feb 2025 12:38:10 +0000 Subject: [PATCH 09/68] Olive.Aws.Ses.AutoFetch.TestConsole doc --- .../Olive.Aws.Ses.AutoFetch.TestConsole.md | 49 +++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 docs/Services/Olive.Aws.Ses.AutoFetch.TestConsole.md diff --git a/docs/Services/Olive.Aws.Ses.AutoFetch.TestConsole.md b/docs/Services/Olive.Aws.Ses.AutoFetch.TestConsole.md new file mode 100644 index 000000000..4300f0415 --- /dev/null +++ b/docs/Services/Olive.Aws.Ses.AutoFetch.TestConsole.md @@ -0,0 +1,49 @@ +# Olive.Aws.Ses.AutoFetch.TestConsole + +## Overview +The `Program` class in this project is responsible for continuously fetching emails from an AWS SES mailbox. It sets up the necessary configuration, initializes the required services, and keeps polling for new emails. + +## Configuration +Ensure the following configurations are set in your `appsettings.json`: +- **ConnectionStrings** - Required for initializing data access. + +## Class: `Program` + +### `Main` +- **Summary**: Initializes the application and starts the email fetching process. +- **Notes**: + - Calls `Init()` to configure the application. + - Starts watching the mailbox using `Mailbox.Watch("my-mailbox")`. + - Continuously fetches new emails every second. + +### `Init` +- **Summary**: Initializes the dependency injection container and database configuration. +- **Steps Performed**: + 1. Loads `appsettings.json`. + 2. Configures logging to output to the console. + 3. Registers database-related services. + 4. Initializes the global `Context` for dependency resolution. + 5. Configures the database connection and access settings. + +## Email Fetching Process + +### `Mailbox.Watch` +- **Summary**: Starts monitoring the specified AWS SES mailbox for new messages. +- **Usage**: +```csharp +await Mailbox.Watch("my-mailbox"); +``` +- **Notes**: + - Automatically detects new emails arriving in the mailbox. + +### `Mailbox.FetchAll` +- **Summary**: Fetches all emails from the monitored mailbox. +- **Usage**: +```csharp +await Mailbox.FetchAll(); +``` +- **Notes**: + - Runs inside an infinite loop, fetching new messages every second. + +## Conclusion +The `Program` class sets up a fully automated AWS SES email fetching system. It continuously monitors and retrieves emails, leveraging dependency injection for service management. Ensure proper database configuration and AWS SES mailbox setup before running the application. \ No newline at end of file From 6ea14db1ab9ac5f91e968bf63aed93b74a70f24e Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Thu, 27 Feb 2025 07:35:43 +0000 Subject: [PATCH 10/68] Olive.Aws.Ses.AutoFetch doc --- docs/Services/Olive.Aws.Ses.AutoFetch.md | 194 +++++++++++++++++++++++ 1 file changed, 194 insertions(+) create mode 100644 docs/Services/Olive.Aws.Ses.AutoFetch.md diff --git a/docs/Services/Olive.Aws.Ses.AutoFetch.md b/docs/Services/Olive.Aws.Ses.AutoFetch.md new file mode 100644 index 000000000..b55a2cd88 --- /dev/null +++ b/docs/Services/Olive.Aws.Ses.AutoFetch.md @@ -0,0 +1,194 @@ +# Olive.Aws.Ses.AutoFetch + +## Overview +The `Olive.Aws.Ses.AutoFetch` provides a mechanism for automatically fetching emails from AWS Simple Email Service (SES) stored in an S3 bucket. It supports processing emails, extracting attachments, and saving them in a structured database. + +## Setting Up the Bucket +Refer to [AWS SES Documentation](https://aws.amazon.com/premiumsupport/knowledge-center/ses-receive-inbound-emails/) for setting up the bucket. + +In `Startup.cs`, initialize the fetching engine: + +```csharp +using Olive.Aws.Ses.AutoFetch; + +public override void Configure(IApplicationBuilder app) +{ + .... + // Option 1: Use the built-in default type of AutoFetch.MailMessage. + await Mailbox.Watch("s3 bucket name where SES is pointing to"); + + // Option 2: Use your own type, which should implement AutoFetch.IMailMessage & AutoFetch.IMailMessageAttachment. + await Mailbox.Watch("s3 bucket name where SES is pointing to"); +} +``` + +### Note +If **Option 1** is used, upon application startup, the following database tables will be automatically created: + +```sql +CREATE TABLE [MailMessage] ... +CREATE TABLE [MailMessageAttachments] ... +``` + +If **Option 2** is used, add these entities: + +```csharp +class MailMessage : EntityType +{ + public MailMessage() + { + Implements("Olive.Aws.Ses.AutoFetch.IMailMessage"); + + String("From"); + BigString("To"); + String("Bcc"); + String("Cc"); + String("Subject"); + BigString("Body"); + String("Sender"); + DateTime("Date").Mandatory(); + DateTime("DateDownloaded").Mandatory(); + String("Bucket"); + String("MessageId"); + String("ReplyTo"); + + InverseAssociate("Attachments", "MailMessage"); + } +} + +class MailMessageAttachment : EntityType +{ + public MailMessageAttachment() + { + Implements("Olive.Aws.Ses.AutoFetch.IMailMessageAttachment"); + + Associate("MailMessage"); + OpenFile("Attachment"); + } +} +``` + +## Execution +Create a background process to call: + +```csharp +await Olive.Aws.Ses.AutoFetch.Mailbox.FetchAll(); +``` + +## Convert Old Attachments to Physical Files + +### Steps +1. Add the following methods to the `MailMessage` logic class: + +```csharp +public static async Task ConvertOldAttachmentsToPhysicalFile() +{ + var messages = await Database.Of() + .Where(x => x.Attachments.HasValue() && x.Attachments != "[]") + .Top(10).GetList(); + + var newAttachments = new List(); + var updatedMessages = new List(); + + foreach (var message in messages) + { + var attachments = message.GetAttachments(); + + if (attachments.HasAny()) + { + foreach (var (FileName, Data) in attachments) + { + newAttachments.Add(new MailMessageAttachment + { + MailMessageId = message.ID, + Attachment = new Olive.Entities.Blob(Data, FileName) + }); + } + + var cloneMessage = message.Clone(); + cloneMessage.Attachments = null; + updatedMessages.Add(cloneMessage); + } + } + + using (var scope = Database.CreateTransactionScope()) + { + await Database.Save(newAttachments.ToArray()); + await Database.BulkUpdate(updatedMessages.ToArray()); + scope.Complete(); + } +} + +(string FileName, byte[] Data)[] GetAttachments() +{ + return JArray.Parse(Attachments) + .Select(x => (x["FileName"].ToString(), x["Base64"].ToString().ToBytesFromBase64())) + .ToArray(); +} +``` + +2. Create a background process to call: + +```csharp +await Domain.MailMessage.ConvertOldAttachmentsToPhysicalFile(); +``` + +## Class: `Mailbox` + +### `Watch` +```csharp +public static async Task Watch(string emailS3Bucket) +``` +- **Summary**: Monitors the specified S3 bucket for incoming emails and initializes database providers for mail storage. +- **Usage**: + +```csharp +await Mailbox.Watch("my-email-bucket"); +``` + +### `FetchAll` +```csharp +public static Task FetchAll() +``` +- **Summary**: Fetches all emails from the monitored S3 bucket and stores them in the database. +- **Usage**: + +```csharp +await Mailbox.FetchAll(); +``` + +## Class: `DatabaseTableService` + +### `EnsureDatabaseTable` +Ensures that the necessary database tables (`MailMessages`, `MailMessageAttachments`) exist. + +## Class: `FetchClient` +Retrieves email messages from the specified AWS S3 bucket, extracts attachments, and saves them. + +## Class: `MailMessage` + +### Properties +- **From**: Sender of the email. +- **To**: Recipients of the email. +- **Subject**: Subject of the email. +- **Body**: Email content. +- **Date**: Date the email was sent. +- **Bucket**: The S3 bucket where the email is stored. +- **MessageId**: Unique identifier for the email. +- **Attachments**: Associated file attachments. + +## Class: `MailMessageAttachment` + +### Properties +- **MailMessageId**: ID of the associated email. +- **Attachment**: The file attachment data. + +## Usage Example +```csharp +await Mailbox.Watch("my-email-bucket"); +await Mailbox.FetchAll(); +``` + +## Conclusion +The `Olive.Aws.Ses.AutoFetch` simplifies AWS SES email retrieval by monitoring an S3 bucket, processing messages, extracting attachments, and storing them efficiently in a database. Proper configuration of AWS credentials and database connections is required for smooth operation. + From c1feafe1fd2b0de7def69488001ca55925c7163d Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Thu, 27 Feb 2025 09:24:42 +0000 Subject: [PATCH 11/68] Olive.Aws.Ses doc --- docs/Services/Olive.Aws.Ses.md | 74 ++++++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100644 docs/Services/Olive.Aws.Ses.md diff --git a/docs/Services/Olive.Aws.Ses.md b/docs/Services/Olive.Aws.Ses.md new file mode 100644 index 000000000..97245b71a --- /dev/null +++ b/docs/Services/Olive.Aws.Ses.md @@ -0,0 +1,74 @@ +# Olive.Aws.Ses + +## Overview +The `Olive.Aws.Ses` provides functionality for sending emails using Amazon Simple Email Service (SES). It converts `MailMessage` objects into MIME messages and dispatches them using AWS SES. + +## Configuration +Ensure you have the necessary AWS credentials and SES permissions configured. + +## Class: `AwsSesEmailDispatcher` + +### `Dispatch` +```csharp +public async Task Dispatch(MailMessage mail, IEmailMessage iEmailMessage) +``` +- **Summary**: Sends an email using AWS SES. +- **Usage**: +```csharp +IEmailMessage iEmailMessage = ?? + +var mail = new MailMessage("from@example.com", "to@example.com") +{ + Subject = "Test Email", + Body = "Hello, this is a test email." +}; +await new AwsSesEmailDispatcher().Dispatch(mail, iEmailMessage); +``` +- **Notes**: + - Converts the `MailMessage` to a `MimeMessage`. + - Calls AWS SES API to send the email. + - Supports calendar event attachments (`VCalendarView`). + - Adds file attachments to the email. + - Throws an exception if the email fails to send. + + +## Class: `Extensions` + +### `ToRawMessage` +```csharp +internal static RawMessage ToRawMessage(this MimeMessage message) +``` +- **Summary**: Converts a `MimeMessage` into an AWS SES-compatible `RawMessage`. +- **Usage**: +```csharp +var rawMessage = mimeMessage.ToRawMessage(); +``` + +## Class: `IServiceCollectionExtension` + +### `AddAwsSesProvider` +```csharp +public static IServiceCollection AddAwsSesProvider(this IServiceCollection services) +``` +- **Summary**: Registers `AwsSesEmailDispatcher` as the default email dispatcher in dependency injection. +- **Usage**: +```csharp +services.AddAwsSesProvider(); +``` + +## Full Example +```csharp +var dispatcher = Context.Current.GetService(); + +IEmailMessage iEmailMessage = ?? +var mail = new MailMessage("from@example.com", "to@example.com") +{ + Subject = "Hello World", + Body = "This is a test email." +}; + +await dispatcher.Dispatch(mail, iEmailMessage); +``` + +## Conclusion +The `Olive.Aws.Ses` provides a simple and effective way to send emails using AWS SES. It supports attachments, calendar invites, and integrates with dependency injection for easy use in applications. From 6f8dd10475e77ca2bea7e0069685a4cbac1fab39 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Thu, 27 Feb 2025 16:12:45 +0000 Subject: [PATCH 12/68] Olive.Aws.Textract doc --- docs/Services/Olive.Aws.Textract.md | 117 ++++++++++++++++++++++++++++ 1 file changed, 117 insertions(+) create mode 100644 docs/Services/Olive.Aws.Textract.md diff --git a/docs/Services/Olive.Aws.Textract.md b/docs/Services/Olive.Aws.Textract.md new file mode 100644 index 000000000..fad42e290 --- /dev/null +++ b/docs/Services/Olive.Aws.Textract.md @@ -0,0 +1,117 @@ +# Olive.Aws.Textract + +## Overview +The `Olive.Aws.Textract` provides functionality for extracting text from documents using AWS Textract. It supports text extraction from S3 buckets and in-memory documents, as well as asynchronous processing for large documents. + +## Configuration +Ensure the following configurations are set in your application: +- **AWS:Rekognition:S3Bucket** - The bucket where documents are stored. +- **AWS:Rekognition:S3OutputBucket** - The bucket where output data is stored. + +## AWS Clients +```csharp +public static IAmazonTextract Client +``` +- **Client**: Provides access to AWS Textract operations. + +### `Region(Amazon.RegionEndpoint region)` +```csharp +public static void Region(Amazon.RegionEndpoint region) +``` +- **Summary**: Creates and uses a new AWS Textract client for the specified region. +- **Usage**: +```csharp +AmazonTextract.Region(RegionEndpoint.USEast1); +``` + +## Text Extraction Operations + +### `ExtractTextString` +```csharp +public static async Task ExtractTextString(string documentKey) +``` +- **Summary**: Extracts text as a single string from a document stored in S3. +- **Usage**: +```csharp +var text = await AmazonTextract.ExtractTextString("document.pdf"); +``` + +### `ExtractTextBlocks` +```csharp +public static async Task> ExtractTextBlocks(string documentKey) +``` +- **Summary**: Extracts text as structured blocks from a document stored in S3. +- **Usage**: +```csharp +var blocks = await AmazonTextract.ExtractTextBlocks("document.pdf"); +``` + +### `StartExtraction` +```csharp +public static async Task StartExtraction(string documentKey, string prefix) +``` +- **Summary**: Starts an asynchronous text extraction job for a document in S3. +- **Usage**: +```csharp +var jobId = await AmazonTextract.StartExtraction("document.pdf", "prefix"); +``` + +### `GetJobResultBlocks` +```csharp +public static async Task GetJobResultBlocks(string jobId, string nextToken = null) +``` +- **Summary**: Retrieves block results from an asynchronous text extraction job. +- **Usage**: +```csharp +var results = await AmazonTextract.GetJobResultBlocks(jobId); +``` + +### `GetJobResultText` +```csharp +public static async Task GetJobResultText(string jobId, string nextToken = null) +``` +- **Summary**: Retrieves extracted text results from an asynchronous text extraction job. +- **Usage**: +```csharp +var textResults = await AmazonTextract.GetJobResultText(jobId); +``` + +### `StartAnalyzeDocument` +```csharp +public static async Task StartAnalyzeDocument(string documentKey, string prefix) +``` +- **Summary**: Starts an analysis job to extract structured data (e.g., forms, tables) from a document in S3. +- **Usage**: +```csharp +var jobId = await AmazonTextract.StartAnalyzeDocument("document.pdf", "prefix"); +``` + +### `GetAnalyzeJobResultBlocks` +```csharp +public static async Task GetAnalyzeJobResultBlocks(string jobId, string nextToken = null) +``` +- **Summary**: Retrieves structured block results from an asynchronous document analysis job. +- **Usage**: +```csharp +var analysisBlocks = await AmazonTextract.GetAnalyzeJobResultBlocks(jobId); +``` + +### `GetAnalyzeJobResultText` +```csharp +public static async Task GetAnalyzeJobResultText(string jobId, string nextToken = null) +``` +- **Summary**: Retrieves extracted text from an asynchronous document analysis job. +- **Usage**: +```csharp +var analysisText = await AmazonTextract.GetAnalyzeJobResultText(jobId); +``` + +## Full Example +```csharp +AmazonTextract.Region(RegionEndpoint.USEast1); +var text = await AmazonTextract.ExtractTextString("document.pdf"); +Console.WriteLine(text); +``` + +## Conclusion +The `Olive.Aws.Textract` provides a simple and effective way to extract text from documents using AWS Textract. It supports synchronous and asynchronous processing for efficient text retrieval. \ No newline at end of file From 5c92716561d2aa37d3aa0ae105498ebd4267fd5b Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Fri, 28 Feb 2025 07:39:19 +0000 Subject: [PATCH 13/68] Olive.Aws doc --- docs/Services/Olive.Aws.md | 203 +++++++++++++++++++++++++++++++++++++ 1 file changed, 203 insertions(+) create mode 100644 docs/Services/Olive.Aws.md diff --git a/docs/Services/Olive.Aws.md b/docs/Services/Olive.Aws.md new file mode 100644 index 000000000..549a0ef77 --- /dev/null +++ b/docs/Services/Olive.Aws.md @@ -0,0 +1,203 @@ +# Olive.Aws + +## Overview +Olive provides first-class integration with several AWS services, enabling seamless authentication and interaction with AWS APIs. You can also directly use any AWS service via the [AWS SDK for .NET](https://aws.amazon.com/sdk-for-net/). + +This document covers the built-in AWS plugins in Olive, authentication mechanisms, role management, secret handling, and best practices. + +--- + +## Built-in AWS Plugins in Olive +Olive includes several AWS-based implementations for common functionalities: + +- **Olive.Aws.EventBus**: Implements `EventBus` using AWS SQS. +- **Olive.Aws.Ses**: Implements `EmailService` using AWS SES. +- **Olive.BlobAws**: Implements `Blob` storage using AWS S3. +- **Olive.Security.Aws**: Implements `.NET Core's IDataProtector` using AWS KMS. +- **Olive.Aws**: A core enabler component that manages authentication and connection setup for AWS API clients. + +--- + +## AWS API Client Authentication +To access AWS APIs, your application must authenticate using an AWS IAM user or IAM role. It is recommended to use **IAM Roles** for flexibility and security. + +### Authentication Methods +Olive supports three authentication methods: + +### 1. API Key Pair (Development) +For local development, store AWS credentials in your `appSettings.Development.json`: + +```json +{ + "Aws": { + "Credentials": { + "AccessKey": "...", + "Secret": "..." + } + } +} +``` + +### 2. EC2 Instance Role (Production) +On production EC2 instances, IAM roles are attached directly, eliminating the need to manage credentials in config files. + +### 3. EC2 Instance + Assume Role (For Multi-App Environments) +When multiple applications run on the same EC2 instance, define a role for the instance and separate roles for each application. Use `RuntimeIdentity` in Olive to assume application-specific roles dynamically: + +```csharp +config.LoadAwsIdentity(); +``` + +Switch between these authentication methods in `Startup.cs`: + +```csharp +public Startup(IWebHostEnvironment env, IConfiguration config, ILoggerFactory factory) + : base(env, config, factory) +{ + if (env.IsProduction()) config.LoadAwsIdentity(); + else config.LoadAwsDevIdentity(); +} +``` + +--- + +## AssumeRole Class + +### Purpose +The `AssumeRole` class provides methods to: +- Assume an AWS IAM role and set it as the default application identity. +- Generate temporary credentials for a given AWS role. +- Automatically renew assumed role credentials in the background. + +### Usage + +#### Assume a Role +```csharp +await AssumeRole.Assume("arn:aws:iam::123456789012:role/MyRole"); +``` +This will change the default application identity to the specified role in the specified AWS account. + +#### Get Temporary Credentials +```csharp +var credentials = await AssumeRole.Temporary("arn:aws:iam::123456789012:role/MyRole"); +``` +This method returns temporary AWS credentials for the specified role without changing the application's identity. + +#### Automatically Renew Credentials +```csharp +AssumeRole.KeepRenewing(); +``` +This starts a background process that renews the assumed role credentials every 5 minutes. + +### Methods + +#### `Task Assume(string roleArn)` +Sets the default application identity to the specified AWS IAM role and renews the credentials. + +#### `Task Temporary(string roleArn)` +Returns temporary credentials for the given IAM role without setting it as the default identity. + +#### `Task SignalRenew()` +Checks if credentials need renewal and renews them if they are older than 5 minutes. + +#### `void KeepRenewing()` +Starts an infinite background process that renews credentials every 5 minutes. + +#### `static async Task Renew()` +Internally used to renew credentials and reset AWS credential factories. + +#### `static async Task LoginAs(string roleArn)` +Handles the actual `AssumeRole` AWS API request and logs the response status. + +## AWS Extensions (`AWSExtensions` Class) + +### Purpose +This class provides extension methods for `IConfiguration` to: +- Load AWS identity from environment variables or application settings. +- Load AWS credentials for development environments. +- Retrieve AWS Secrets from AWS Secrets Manager or AWS Systems Manager Parameter Store. + +### Usage + +#### Load AWS Identity from Environment Variables +```csharp +Configuration.LoadAwsIdentity(); +``` +This loads AWS credentials and secrets from environment variables. +This is intended for use in Production under Kubernetes pods as a workaround for adding AWS Roles to specific pods as opposed to the host server. +This is not needed in Lambda as you can allocate a role to lambda directly. + +#### Load AWS Identity for Development +```csharp +Configuration.LoadAwsDevIdentity(); +``` +Loads credentials from `appsettings.json` under `Aws:Credentials:AccessKey` and `Aws:Credentials:Secret`. +Use this if you want to have AWS calls made under the current machine's Role,or the user specified in appSettings under (Aws { Credentials { AccessKey: ... , Secret: ... } }). + +#### Load AWS Secrets +```csharp +Configuration.LoadAwsSecrets(SecretProviderType.SecretsManager); +``` +Loads AWS secrets from AWS Secrets Manager. + +### Methods + +#### `void LoadAwsIdentity()` +Loads AWS credentials from environment variables and application settings. + +#### `void LoadAwsDevIdentity(bool loadSecrets = false)` +Loads AWS credentials from `appsettings.json` and optionally loads AWS secrets. + +#### `void LoadAwsDevIdentity(string accessKey, string secret, RegionEndpoint endpoint, bool loadSecrets)` +Explicitly loads AWS credentials using provided access key and secret. + +#### `void LoadAwsSecrets(SecretProviderType provider = SecretProviderType.SecretsManager)` +Loads AWS secrets using the specified provider (Secrets Manager or Systems Manager Parameter Store). + +## Runtime Identity (`RuntimeIdentity` Class) + +### Purpose +Specifies the runtime IAM identity for an application and assumes the configured AWS role. + +### Usage +```csharp +await RuntimeIdentity.Load(Configuration); +``` +Loads the AWS IAM role from environment variables and assumes it. + +### Methods + +#### `static async Task Load(IConfiguration config)` +Loads and assumes the AWS IAM role specified in the environment variable `AWS_RUNTIME_ROLE_ARN`. + +## Secrets Management (`Secrets` Class) + +### Purpose +Handles fetching AWS Secrets from AWS Secrets Manager or Systems Manager Parameter Store. + +### Usage +```csharp +var secrets = new Secrets(Configuration, SecretProviderType.SecretsManager); +secrets.Load(); +``` + +### Methods + +#### `protected override string DownloadSecrets()` +Retrieves secrets using the configured AWS secrets provider. + +## Enums + +### `SecretProviderType` +Defines the AWS secrets provider type: +- `SecretsManager`: AWS Secrets Manager +- `SystemsManagerParameter`: AWS Systems Manager Parameter Store + +## Important Considerations + +- **Security:** Do not hardcode AWS access keys and secrets. Always use IAM roles, environment variables, or AWS Secrets Manager. +- **Role Expiry:** AWS STS tokens expire. The application must handle token renewal using `KeepRenewing()`. +- **Error Handling:** Errors while assuming roles or fetching secrets should be logged and handled appropriately. + +This document provides an overview of the AWS Assume Role and identity management utilities, ensuring secure and scalable authentication for AWS services. + From 1724c728d3d3b20ca860a8169318eb0397954b16 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Fri, 28 Feb 2025 08:31:34 +0000 Subject: [PATCH 14/68] Olive.Azure.DocumentAnalyzer doc --- docs/Azure/Olive.Azure.DocumentAnalyzer.md | 148 +++++++++++++++++++++ 1 file changed, 148 insertions(+) create mode 100644 docs/Azure/Olive.Azure.DocumentAnalyzer.md diff --git a/docs/Azure/Olive.Azure.DocumentAnalyzer.md b/docs/Azure/Olive.Azure.DocumentAnalyzer.md new file mode 100644 index 000000000..9c18644ad --- /dev/null +++ b/docs/Azure/Olive.Azure.DocumentAnalyzer.md @@ -0,0 +1,148 @@ +**Olive.Azure.DocumentAnalyzer** + +## Overview +The `Olive.Azure.DocumentAnalyzer` provides methods to interact with Azure Document Intelligence API to analyze documents, list models, and build document analyzers. + +## Configuration +Ensure that the following configuration values are set before using the API: +- `Azure:DocumentIntelligence:Endpoint` - The endpoint URL for the Azure Document Intelligence service. +- `Azure:DocumentIntelligence:ApiKey` - The API key required for authentication. +- `Azure:DocumentIntelligence:ApiVersion` - The version of the API being used. + +## Methods + +### `BuildDocumentAnalyzer` +**Description:** +Creates and trains a document model using files stored in an Azure Blob Storage container. + +**Signature:** +```csharp +public async Task BuildDocumentAnalyzer( + string modelId, + string trainingFolderName, + string description = null, + string storageContainerName = null) +``` + +**Parameters:** +- `modelId` (string) - The identifier for the model. +- `trainingFolderName` (string) - The folder containing training documents. +- `description` (string, optional) - A description of the model. +- `storageContainerName` (string, optional) - The name of the Azure Storage container. + +**Returns:** +- `BuildDocumentAnalyzerResponse` - Contains details about the document analysis operation. + +**Usage:** +```csharp +var result = await documentAnalyze.BuildDocumentAnalyzer("model123", "training-folder"); +``` + +--- + +### `ListDocumentAnalyzeModelsAsync` +**Description:** +Retrieves a list of available document models from Azure Document Intelligence. + +**Signature:** +```csharp +public async Task ListDocumentAnalyzeModelsAsync( + bool skipPrebuilt = true, + bool useFormRecognizer = false) +``` + +**Parameters:** +- `skipPrebuilt` (bool) - If true, excludes prebuilt models. +- `useFormRecognizer` (bool) - If true, uses Form Recognizer instead of Document Intelligence. + +**Returns:** +- `DocumentAnalyzeModelResponse` - A list of available models. + +**Usage:** +```csharp +var models = await documentAnalyze.ListDocumentAnalyzeModelsAsync(); +``` + +--- + +### `AnalyzeDocumentAsync` +**Description:** +Analyzes a document using a specified model. + +**Signature:** +```csharp +public async Task AnalyzeDocumentAsync( + byte[] fileData, + string modelId, + bool useFormRecognizer = false) +``` + +**Parameters:** +- `fileData` (byte[]) - The document data in byte format. +- `modelId` (string) - The ID of the document model to use. +- `useFormRecognizer` (bool) - If true, uses Form Recognizer instead of Document Intelligence. + +**Returns:** +- `string` - The operation location URL to check the analysis results. + +**Usage:** +```csharp +var resultUrl = await documentAnalyze.AnalyzeDocumentAsync(fileBytes, "model123"); +``` + +--- + +### `AnalyzeDocumentAsyncWithPoll` +**Description:** +Analyzes a document and polls for the results until completion. + +**Signature:** +```csharp +public async Task AnalyzeDocumentAsyncWithPoll( + byte[] fileData, + string modelId, + bool useFormRecognizer = false) +``` + +**Returns:** +- `DocumentAnalyzeResult` - Contains extracted data from the document. + +**Usage:** +```csharp +var analysisResult = await documentAnalyze.AnalyzeDocumentAsyncWithPoll(fileBytes, "model123"); +``` + +--- + +### `GetPollResults` +**Description:** +Fetches the results of a previously started document analysis operation. + +**Signature:** +```csharp +public async Task GetPollResults(string resultUrl) +``` + +**Returns:** +- `DocumentAnalyzeResult` - The final document analysis results. + +**Usage:** +```csharp +var result = await documentAnalyze.GetPollResults(operationUrl); +``` + +--- + +## Error Handling +- If an API request fails, an exception will be thrown with the corresponding HTTP status code and error message. +- Network failures or incorrect API keys may cause authentication errors. +- Ensure that the provided model ID and training data exist in the configured Azure Blob Storage. + +## Notes +- The class interacts with Azure's Document Intelligence API, requiring a valid API key. +- Polling methods wait until the document processing completes before returning results. +- Ensure that the `endpoint` and `apiKey` are correctly configured before making API calls. + +--- +This documentation provides a structured reference for developers integrating with the `Olive.Azure.DocumentAnalyzer`. If further details are needed, refer to the official Azure Document Intelligence API documentation. + From 779d9799d036620b6db0cbb6814cc0ff420a3958 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Fri, 28 Feb 2025 08:57:16 +0000 Subject: [PATCH 15/68] Olive.Azure.DocumentClassification doc --- .../Olive.Azure.DocumentClassification.md | 101 ++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 docs/Azure/Olive.Azure.DocumentClassification.md diff --git a/docs/Azure/Olive.Azure.DocumentClassification.md b/docs/Azure/Olive.Azure.DocumentClassification.md new file mode 100644 index 000000000..4c67e32e0 --- /dev/null +++ b/docs/Azure/Olive.Azure.DocumentClassification.md @@ -0,0 +1,101 @@ +**# Olive.Azure.DocumentClassification** + +## Overview +The `Olive.Azure.DocumentClassification` provides methods to interact with the Azure Document Intelligence service. It enables listing available classifiers and classifying documents either from a local file or a remote URL. + +--- + +## Configuration +Ensure the following configuration settings are available in your application settings: +```json +{ + "Azure": { + "Classification": { + "Endpoint": "", + "ApiKey": "", + "ModelId": "", + "ApiVersion": "" + } + } +} +``` + +--- + +## Methods + +### `Task ListDocumentClassifiersAsync(bool useFormRecognizer = false)` +#### Description +Retrieves a list of available document classifiers from Azure Document Intelligence. + +#### Parameters +- `useFormRecognizer` *(bool, optional)* – If `true`, uses the Form Recognizer API instead of Document Intelligence. + +#### Returns +- `DocumentClassifierResponse`: A list of available document classifiers. + +#### Usage Example +```csharp +var classifier = new DocumentClassifier(); +var classifiers = await classifier.ListDocumentClassifiersAsync(); +``` + +--- + +### `Task ClassifyDocumentAsync(byte[] fileData, string modelId)` +#### Description +Classifies a document from a given file data using a specified model ID. + +#### Parameters +- `fileData` *(byte[])* – The document file in byte array format. +- `modelId` *(string)* – The classifier model ID to use. + +#### Returns +- `DocumentClassifierResult`: The classification result containing document type and confidence score. + +#### Usage Example +```csharp +byte[] fileBytes = File.ReadAllBytes("document.pdf"); +var classifier = new DocumentClassifier(); +var result = await classifier.ClassifyDocumentAsync(fileBytes, "myModelId"); +``` + +--- + +### `Task ClassifyDocumentAsync(string filePathOrURI, string modelId)` +#### Description +Classifies a document from a given file path or remote URL. + +#### Parameters +- `filePathOrURI` *(string)* – The path to the document file or a URL. +- `modelId` *(string)* – The classifier model ID to use. + +#### Returns +- `DocumentClassifierResult`: The classification result. + +#### Usage Example +```csharp +var classifier = new DocumentClassifier(); +var result = await classifier.ClassifyDocumentAsync("https://example.com/document.pdf", "myModelId"); +``` + +--- + +## Supporting Classes + +### `DocumentClassifierResponse` +#### Properties +- `List Value` – The list of classifier models available. + +### `DocumentClassifierResult` +#### Properties +- `string DocumentType` – The classified document type. +- `decimal Confidence` – The confidence score of the classification. + +--- + +## Notes +- This class uses Azure Document Intelligence API to classify documents. +- Ensure valid API keys and endpoints are configured before using the service. +- If an invalid file path or URL is provided, an exception will be thrown. + From 631ed172f24fb89b2f2fcfc6ef94e72d7a310b97 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Fri, 28 Feb 2025 09:59:14 +0000 Subject: [PATCH 16/68] Olive.Azure.EventBus doc --- docs/Azure/Olive.Azure.EventBus.md | 153 +++++++++++++++++++++++++++++ 1 file changed, 153 insertions(+) create mode 100644 docs/Azure/Olive.Azure.EventBus.md diff --git a/docs/Azure/Olive.Azure.EventBus.md b/docs/Azure/Olive.Azure.EventBus.md new file mode 100644 index 000000000..6cbfd6d01 --- /dev/null +++ b/docs/Azure/Olive.Azure.EventBus.md @@ -0,0 +1,153 @@ +**Olive.Azure.EventBus** + +## Overview +The `Olive.Azure.EventBus` is designed to interact with Azure Service Bus, providing functionalities for publishing, subscribing, and retrieving messages from a queue. This implementation ensures reliable message handling with configurable limits. + +--- + +## **EventBusQueue Class** + +### **Properties** +- `MaxNumberOfMessages`: Defines the maximum number of messages to return from the queue. Defaults to `10`. +- `QueueUrl`: Stores the queue’s URL. +- `IsFifo`: Indicates if the queue is FIFO-based. +- `Limiter`: A rate limiter to control message publishing frequency. + +### **Methods** + +#### **`EventBusQueue(string queueUrl)`** +- Initializes a new instance of the `EventBusQueue` class. +- **Parameters:** + - `queueUrl` *(string)*: The URL of the Azure Service Bus queue. +- **Usage:** + ```csharp + var queue = new EventBusQueue("your-queue-url"); + ``` + +#### **`Task Publish(string message)`** +- Publishes a single message to the Azure Service Bus queue. +- **Parameters:** + - `message` *(string)*: The message content to be sent. +- **Returns:** + - `Task`: The message ID of the published message. +- **Usage:** + ```csharp + string messageId = await queue.Publish("Hello, Azure Service Bus!"); + ``` + +#### **`Task> PublishBatch(IEnumerable messages)`** +- Publishes a batch of messages to the queue. +- **Parameters:** + - `messages` *(IEnumerable)*: Collection of messages to be sent. +- **Returns:** + - `Task>`: A collection of message IDs. +- **Usage:** + ```csharp + var messageIds = await queue.PublishBatch(new List{ "Msg1", "Msg2" }); + ``` + +#### **`void Subscribe(Func handler)`** +- Subscribes to the queue to process incoming messages. +- **Parameters:** + - `handler` *(Func)*: A function to process incoming messages. +- **Usage:** + ```csharp + queue.Subscribe(async message => Console.WriteLine("Received: " + message)); + ``` + +#### **`Task> PullBatch(int timeoutSeconds = 10, int? maxNumerOfMessages = null)`** +- Retrieves a batch of messages from the queue. +- **Parameters:** + - `timeoutSeconds` *(int, optional)*: Timeout for retrieving messages. + - `maxNumerOfMessages` *(int?, optional)*: Maximum number of messages to pull. +- **Returns:** + - `Task>`: Collection of queue message handles. +- **Usage:** + ```csharp + var messages = await queue.PullBatch(); + ``` + +#### **`Task Purge()`** +- Deletes all messages from the queue. +- **Usage:** + ```csharp + await queue.Purge(); + ``` + +--- + +## **Subscriber Class** +The `Subscriber` class handles polling and message processing from an `EventBusQueue`. + +### **Methods** + +#### **`Subscriber(EventBusQueue queue, Func handler)`** +- Initializes a new instance of the `Subscriber` class. +- **Parameters:** + - `queue` *(EventBusQueue)*: The event bus queue instance. + - `handler` *(Func)*: Function to process received messages. + +#### **`void Start()`** +- Starts the polling process for retrieving messages. + +#### **`Task PullAll()`** +- Retrieves all available messages from the queue. + +--- + +## **EventBusProvider Class** +The `EventBusProvider` manages event bus queue instances. + +### **Methods** +#### **`IEventBusQueue Provide(string queueUrl)`** +- Retrieves or creates a queue instance. +- **Usage:** + ```csharp + var provider = new EventBusProvider(); + var queue = provider.Provide("your-queue-url"); + ``` + +--- + +## **AzureMessagingContext Class** +Handles Azure Service Bus client interactions. + +### **Properties** +- `Sender`: Provides a `ServiceBusSender` for publishing messages. +- `Receiver`: Provides a `ServiceBusReceiver` for retrieving messages. +- `Purger`: Provides a `ServiceBusReceiver` for purging messages. + +### **Methods** +#### **`AzureMessagingContext(string queueUrl)`** +- Initializes a messaging context for a given queue. +- **Usage:** + ```csharp + using var context = new AzureMessagingContext("your-queue-url"); + ``` + +--- + +## **AzureEventBusExtensions Class** +Provides an extension method for registering Azure event bus services. + +### **Methods** +#### **`IServiceCollection AddAzureEventBus(this IServiceCollection services)`** +- Registers `IEventBusQueueProvider` with the DI container. +- **Usage:** + ```csharp + services.AddAzureEventBus(); + ``` + +--- + +## **Warnings & Notes** +- Ensure your Azure Service Bus connection string is correctly configured. +- **Purge method should be used with caution** as it will delete all messages in the queue. +- FIFO queues should be handled carefully to maintain ordering. +- Implement proper exception handling when processing messages. + +--- + +## **Conclusion** +The `EventBusQueue` provides a robust solution for integrating Azure Service Bus into your applications. With capabilities for message publishing, batch processing, and automatic message polling, it simplifies event-driven communication in distributed systems. + From 865704e0b148fc96a3ed2af06fcbccb197576dae Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Fri, 28 Feb 2025 10:17:34 +0000 Subject: [PATCH 17/68] Olive.Azure doc --- docs/Azure/Olive.Azure.md | 56 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) create mode 100644 docs/Azure/Olive.Azure.md diff --git a/docs/Azure/Olive.Azure.md b/docs/Azure/Olive.Azure.md new file mode 100644 index 000000000..9cae3c759 --- /dev/null +++ b/docs/Azure/Olive.Azure.md @@ -0,0 +1,56 @@ +# Olive.Azure + +## Overview +This documentation explains how to use Azure App Configuration for managing secrets using the `Secrets` class in the `Olive.Azure` namespace. The configuration secrets are retrieved from Azure App Configuration and loaded into the application settings. + +## Configuration +Before using the `Secrets` class, ensure that the following settings are correctly configured in your application: + +- **Azure:Secrets:Key** - The key identifier for the secret in Azure App Configuration. +- **Azure:Secrets:Endpoint** - The endpoint URL for your Azure App Configuration instance. + +Example configuration in `appsettings.json`: +```json +{ + "Azure": { + "Secrets": { + "Key": "MySecretKey", + "Endpoint": "https://my-app-config.azconfig.io" + } + } +} +``` + +## Secrets Class +The `Secrets` class is responsible for retrieving secrets stored in Azure App Configuration. + +--- + +## Azure Extensions +The `AzureExtensions` class provides additional methods for working with Azure configurations. + +### **Methods** +#### `LoadAzureSecrets` +```csharp +public static void LoadAzureSecrets(this IConfiguration @this) +``` +Loads secret information from Azure App Configuration. This method initializes an instance of the `Secrets` class and calls `Load()` to retrieve secrets. + +Example usage: +```csharp +IConfiguration configuration = new ConfigurationBuilder().Build(); +configuration.LoadAzureSecrets(); +``` + +## Error Handling +If the necessary settings are missing, the following exceptions may occur: +- `ArgumentNullException` if `Azure:Secrets:Key` or `Azure:Secrets:Endpoint` is not found. +- `RequestFailedException` if there is an issue with retrieving secrets from Azure. + +## Notes +- Ensure that your application has the correct Azure Identity permissions to access the App Configuration service. +- Use Managed Identity or service principals for authentication. + +## Conclusion +The `Secrets` class and `LoadAzureSecrets` method provide a seamless way to integrate Azure App Configuration secrets into your application, ensuring secure and centralized configuration management. + From 411adfa34979ee616290615a42fe18945a95486f Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Fri, 28 Feb 2025 11:48:26 +0000 Subject: [PATCH 18/68] Olive.AzureSearch doc --- docs/Azure/Olive.AzureSearch.md | 112 ++++++++++++++++++++++++++++++++ 1 file changed, 112 insertions(+) create mode 100644 docs/Azure/Olive.AzureSearch.md diff --git a/docs/Azure/Olive.AzureSearch.md b/docs/Azure/Olive.AzureSearch.md new file mode 100644 index 000000000..c7381d320 --- /dev/null +++ b/docs/Azure/Olive.AzureSearch.md @@ -0,0 +1,112 @@ +**# Olive.AzureSearch** + +## Overview +`Olive.AzureSearch` is a repository designed to work with Azure Cognitive Search. It provides functionalities to manage search filters, push data, retrieve search results, and remove search filters. + +--- + +## **Class: AzureSearchFilterRepository** + +### **Constructor** +```csharp +public AzureSearchFilterRepository(AzureSearchConfigModel configuration) +``` +- Initializes the repository with an Azure search index based on the provided `AzureSearchConfigModel`. + +### **Methods** + +#### **Clear()** +```csharp +public void Clear() +``` +- Clears the current index if it exists and reinitializes it. +- Handles exceptions where the index is not found. + +#### **PushSearchFilter(IEnumerable searchFilters)** +```csharp +public void PushSearchFilter(IEnumerable searchFilters) +``` +- Uploads a batch of search filters into the index. +- Uses `IndexDocumentsBatch.Create()` to process multiple documents. + +#### **GetBySearchTerm(string searchTerm, SearchOptions searchOptions = null, int maxCountToAddSuffixResponse = 60)** +```csharp +public IEnumerable GetBySearchTerm(string searchTerm, SearchOptions searchOptions = null, int maxCountToAddSuffixResponse = 60) +``` +- Searches the index using the provided search term. +- If the initial search does not return enough results, it performs an additional search with a suffix pattern. +- Returns distinct results. + +#### **RemoveSearchFilter(string searchTerm, params string[] searchFields)** +```csharp +public bool RemoveSearchFilter(string searchTerm, params string[] searchFields) +``` +- Searches for records matching the given search term and removes them from the index. +- Uses `DeleteDocuments()` to delete matching documents. + +#### **GetDefaultSearchOptions()** +```csharp +public SearchOptions GetDefaultSearchOptions() +``` +- Creates default search options, setting filterable fields and defining query settings. + +--- + +## **Class: AzureSearchConfigModel** + +### **Properties** +```csharp +public string AdminKey { get; set; } +public string ServiceUrl { get; set; } +public bool Enabled { get; set; } +public string DefaultIndex { get; set; } +``` +- Contains configuration settings required for connecting to Azure Cognitive Search. + +--- + +## **Class: AzureSearchFilterDTOBase** + +### **Properties** +```csharp +[SimpleField(IsKey = true)] +public string Id { get; set; } +``` +- Represents the base DTO for Azure search filters. +- Includes an `Id` field marked as a unique key. + +--- + +## **Usage Example** + +```csharp +var config = new AzureSearchConfigModel +{ + AdminKey = "your-admin-key", + ServiceUrl = "https://your-search-service.search.windows.net", + DefaultIndex = "your-index", + Enabled = true +}; + +var repository = new AzureSearchFilterRepository(config); + +// Add filters +repository.PushSearchFilter(new List +{ + new MySearchDTO { Id = "1", Name = "Example" } +}); + +// Retrieve search results +var results = repository.GetBySearchTerm("Example"); + +// Remove a filter +repository.RemoveSearchFilter("Example"); +``` + +--- + +## **Notes** +- Ensure that the Azure Cognitive Search service is correctly configured before using the repository. +- The `AdminKey` must be valid to perform index modifications. +- Search queries support full-text search and wildcard matching. + From 722ef48c9ec0f42d0e76db0cfb9dd362a519bb26 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Fri, 28 Feb 2025 12:39:55 +0000 Subject: [PATCH 19/68] Olive.Blob.Aws doc --- docs/Services/Olive.Blob.Aws.md | 302 ++++++++++++++++++++++++++++++++ 1 file changed, 302 insertions(+) create mode 100644 docs/Services/Olive.Blob.Aws.md diff --git a/docs/Services/Olive.Blob.Aws.md b/docs/Services/Olive.Blob.Aws.md new file mode 100644 index 000000000..e64270abd --- /dev/null +++ b/docs/Services/Olive.Blob.Aws.md @@ -0,0 +1,302 @@ +# Olive.Blob.Aws + +This document outlines the public methods, variables, and properties in the `Olive.BlobAws` namespace, designed to help users interact with Amazon S3 for file storage, retrieval, and management. It assumes you are integrating this package into your project and need to use its public API. + +--- + +## Classes and Public Members + +### 1. `S3PresignedUrlGenerator` +A class to generate presigned URLs for S3 objects, implementing `IS3PresignedUrlGenerator`. + +#### Public Properties +- `DefaultTimeout`: Gets the default expiration time for presigned URLs (read-only, type: `TimeSpan`). + +#### Public Constructors +```csharp +public S3PresignedUrlGenerator() +public S3PresignedUrlGenerator(TimeSpan defaultTimeout) +``` +- **Purpose:** Initializes the generator. + - Default constructor uses a 30-minute timeout. + - Parameterized constructor allows a custom timeout. +- **Parameters:** + - `defaultTimeout`: The custom expiration time for presigned URLs. + +#### Public Methods +##### `Sign(Blob blob, TimeSpan? timeout = null)` +```csharp +public string Sign(Blob blob, TimeSpan? timeout = null) +``` +- **Purpose:** Generates a presigned URL for an S3 object using a `Blob`. +- **Parameters:** + - `blob`: The `Blob` object to generate a URL for. + - `timeout`: Optional expiration time (defaults to `DefaultTimeout`). +- **Returns:** A presigned URL string. + +##### `Sign(string key, TimeSpan? timeout = null)` +```csharp +public string Sign(string key, TimeSpan? timeout = null) +``` +- **Purpose:** Generates a presigned URL for an S3 object using its key. +- **Parameters:** + - `key`: The S3 key of the object. + - `timeout`: Optional expiration time (defaults to `DefaultTimeout`). +- **Returns:** A presigned URL string. + +--- + +### 2. `S3FileRequestService` +A service implementing `IFileRequestService` for S3 file operations. + +#### Public Constructors +```csharp +public S3FileRequestService(FileUploadSettings settings) +``` +- **Purpose:** Initializes the service with S3 settings. +- **Parameters:** + - `settings`: An instance of `FileUploadSettings` with bucket and region details (typically injected via dependency injection). + +#### Public Methods +##### `Bind(string fileKey)` +```csharp +public async Task Bind(string fileKey) +``` +- **Purpose:** Retrieves a file from a temporary S3 bucket and returns it as a `Blob`. +- **Parameters:** + - `fileKey`: The key prefix of the file in the temporary bucket. +- **Returns:** A `Blob` object containing the file content and name. +- **Throws:** + - `Exception` if no file or multiple files are found in the specified folder. + +##### `CreateDownloadAction(byte[] data, string filename)` +```csharp +public async Task CreateDownloadAction(byte[] data, string filename) +``` +- **Purpose:** Uploads a file to S3 and returns a public download URL. +- **Parameters:** + - `data`: The file content as a byte array. + - `filename`: The name of the file. +- **Returns:** An anonymous object with a `Download` property containing the URL. + +##### `DeleteTempFiles(TimeSpan _)` +```csharp +public Task DeleteTempFiles(TimeSpan _) +``` +- **Purpose:** Intended to delete temporary files (not implemented). +- **Parameters:** + - `_`: A `TimeSpan` parameter (ignored). +- **Throws:** `InvalidOperationException` with message "S3 should take care of this." + +##### `Download(string key)` +```csharp +public Task Download(string key) +``` +- **Purpose:** Intended to download a file (not implemented). +- **Parameters:** + - `key`: The S3 key of the file. +- **Throws:** `InvalidOperationException` with message "Client should download from S3." + +##### `TempSaveUploadedFile(IFormFile file, bool allowUnsafeExtension = false)` +```csharp +public Task TempSaveUploadedFile(IFormFile file, bool allowUnsafeExtension = false) +``` +- **Purpose:** Intended to save an uploaded file temporarily (not implemented). +- **Parameters:** + - `file`: The uploaded file. + - `allowUnsafeExtension`: Whether to allow unsafe file extensions (default: `false`). +- **Throws:** `InvalidOperationException` with message "Client should upload to S3 directly." + +--- + +### 3. `S3FileProvider` +Implements `IFileProvider` for accessing S3 as a file system. + +#### Public Constructors +```csharp +public S3FileProvider(IAmazonS3 amazonS3, string bucketName) +``` +- **Purpose:** Initializes the provider with an S3 client and bucket. +- **Parameters:** + - `amazonS3`: An `IAmazonS3` client instance. + - `bucketName`: The name of the S3 bucket. + +#### Public Methods +##### `GetDirectoryContents(string subpath)` +```csharp +public IDirectoryContents GetDirectoryContents(string subpath) +``` +- **Purpose:** Lists the contents of an S3 directory. +- **Parameters:** + - `subpath`: The path within the bucket (relative to the root). +- **Returns:** An `IDirectoryContents` object (e.g., `S3DirectoryContents`). +- **Throws:** `ArgumentNullException` if `subpath` is null. + +##### `GetFileInfo(string subpath)` +```csharp +public IFileInfo GetFileInfo(string subpath) +``` +- **Purpose:** Retrieves metadata for an S3 file. +- **Parameters:** + - `subpath`: The path to the file within the bucket. +- **Returns:** An `IFileInfo` object (e.g., `S3FileInfo`). +- **Throws:** `ArgumentNullException` if `subpath` is null. + +##### `Watch(string filter)` +```csharp +public IChangeToken Watch(string filter) +``` +- **Purpose:** Monitors changes (not supported in S3). +- **Parameters:** + - `filter`: The filter pattern (ignored). +- **Returns:** A `NullChangeToken` (no change monitoring). + +##### `Dispose()` +```csharp +public void Dispose() +``` +- **Purpose:** Disposes the underlying S3 client. + +--- + +### 4. `S3FileInfo` +Implements `IFileInfo` for S3 file metadata. + +#### Public Properties +- `Exists`: `bool` indicating if the file exists in S3. +- `Length`: `long` representing the file size in bytes. +- `Name`: `string` containing the file name. +- `PhysicalPath`: `string` with the S3 URL (e.g., `s3-region.amazonaws.com/bucket/key`). +- `LastModified`: `DateTimeOffset` of the last modification. +- `IsDirectory`: `bool` indicating if the key is a directory (ends with `/`). + +#### Public Methods +##### `CreateReadStream()` +```csharp +public Stream CreateReadStream() +``` +- **Purpose:** Provides a stream to read the file content. +- **Returns:** A `Stream` object. + +--- + +### 5. `S3BlobStorageProvider` +Implements `IBlobStorageProvider` for S3 storage operations. + +#### Public Methods +##### `CostsToCheckExistence()` +```csharp +public bool CostsToCheckExistence() +``` +- **Purpose:** Indicates if checking existence incurs a cost. +- **Returns:** `true` (S3 operations have a cost). + +##### `SaveAsync(Blob document)` +```csharp +public async Task SaveAsync(Blob document) +``` +- **Purpose:** Uploads a `Blob` to S3. +- **Parameters:** + - `document`: The `Blob` to save. +- **Throws:** `Exception` if the upload fails. + +##### `FileExistsAsync(Blob document)` +```csharp +public async Task FileExistsAsync(Blob document) +``` +- **Purpose:** Checks if a `Blob` exists in S3. +- **Parameters:** + - `document`: The `Blob` to check. +- **Returns:** `true` if the file exists, `false` if not. + +##### `LoadAsync(Blob document)` +```csharp +public async Task LoadAsync(Blob document) +``` +- **Purpose:** Loads a `Blob` from S3 as a byte array. +- **Parameters:** + - `document`: The `Blob` to load. +- **Returns:** A byte array of the file content. +- **Throws:** `Exception` with error logging if loading fails. + +##### `DeleteAsync(Blob document)` +```csharp +public async Task DeleteAsync(Blob document) +``` +- **Purpose:** Deletes a `Blob` from S3. +- **Parameters:** + - `document`: The `Blob` to delete. +- **Throws:** `Exception` if deletion fails. + +--- + +### 6. `BlobAWSExtensions` +Extension methods for integrating S3 functionality. + +#### Public Methods +##### `AddS3BlobStorageProvider(this IServiceCollection @this, TimeSpan PresignedUrlTimeout)` +```csharp +public static IServiceCollection AddS3BlobStorageProvider(this IServiceCollection @this, TimeSpan PresignedUrlTimeout) +``` +- **Purpose:** Registers `S3BlobStorageProvider` and `S3PresignedUrlGenerator` with a custom timeout. +- **Parameters:** + - `@this`: The `IServiceCollection` instance. + - `PresignedUrlTimeout`: Timeout for presigned URLs. +- **Returns:** The updated `IServiceCollection`. + +##### `AddS3BlobStorageProvider(this IServiceCollection @this)` +```csharp +public static IServiceCollection AddS3BlobStorageProvider(this IServiceCollection @this) +``` +- **Purpose:** Registers `S3BlobStorageProvider` and `S3PresignedUrlGenerator` with default settings. +- **Parameters:** + - `@this`: The `IServiceCollection` instance. +- **Returns:** The updated `IServiceCollection`. + +##### `AddS3FileRequestService(this IServiceCollection @this)` +```csharp +public static IServiceCollection AddS3FileRequestService(this IServiceCollection @this) +``` +- **Purpose:** Registers S3 file request services (`S3FileRequestService`, `S3FileUploadMarkupGenerator`). +- **Parameters:** + - `@this`: The `IServiceCollection` instance. +- **Returns:** The updated `IServiceCollection`. + +##### `GetS3PresignedUrl(this Blob document, Action config = null)` +```csharp +public static string GetS3PresignedUrl(this Blob document, Action config = null) +``` +- **Purpose:** Generates a presigned URL for a `Blob` with customizable request configuration. +- **Parameters:** + - `document`: The `Blob` to generate a URL for. + - `config`: Optional action to configure the `GetPreSignedUrlRequest`. +- **Returns:** A presigned URL string. + +--- + +## Usage Example +```csharp +// Setup services +services.AddS3BlobStorageProvider(TimeSpan.FromMinutes(60)); +services.AddS3FileRequestService(); + +var provider = Context.Current; + +// Upload a file +var blob = new Blob(File.ReadAllBytes("example.txt"), "example.txt"); +var storageProvider = provider.GetService(); +await storageProvider.SaveAsync(blob); + +// Check existence +var exists = await storageProvider.FileExistsAsync(blob); + +// Load content +var content = await storageProvider.LoadAsync(blob); + +// Generate presigned URL +var urlGenerator = provider.GetService(); +var presignedUrl = urlGenerator.Sign(blob); + +// Delete file +await storageProvider.DeleteAsync(blob); +``` \ No newline at end of file From fb4696786dff5513a1963bc99b5f76d463cf0b0a Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Fri, 28 Feb 2025 14:23:08 +0000 Subject: [PATCH 20/68] Olive.Blob.Azure doc --- docs/Azure/Olive.Blob.Azure.md | 165 +++++++++++++++++++++++++++++++++ 1 file changed, 165 insertions(+) create mode 100644 docs/Azure/Olive.Blob.Azure.md diff --git a/docs/Azure/Olive.Blob.Azure.md b/docs/Azure/Olive.Blob.Azure.md new file mode 100644 index 000000000..f537c9bbc --- /dev/null +++ b/docs/Azure/Olive.Blob.Azure.md @@ -0,0 +1,165 @@ +# Olive.Blob.Azure + +This document outlines the public methods, variables, and properties in the `Olive.Blob.Azure` namespace, designed to help users interact with Azure Blob Storage for file storage, retrieval, and management. + +--- + +### 1. `SASTokenGenerator` +A class to generate SAS (Shared Access Signature) tokens for Azure Blob Storage containers. + +##### `GenerateContainerSasUri(string storageContainerName = null)` +```csharp +public Uri GenerateContainerSasUri(string storageContainerName = null) +``` +- **Purpose:** Generates a SAS URI for an Azure Blob container with read, write, create, add, delete, and list permissions. +- **Parameters:** + - `storageContainerName`: Optional container name (defaults to the configured `AzureBlobInfo.StorageContainer` if not provided). +- **Returns:** A `Uri` object representing the SAS URI, valid for 8 hours from generation. + +--- + +### 2. `BlobAzureExtension` +Extension methods for integrating Azure Blob Storage into a service collection. + +#### Public Methods +##### `AddAzureBlobStorageProvider(this IServiceCollection @this)` +```csharp +public static IServiceCollection AddAzureBlobStorageProvider(this IServiceCollection @this) +``` +- **Purpose:** Registers `AzureBlobStorageProvider` as an `IBlobStorageProvider` in the dependency injection container. +- **Parameters:** + - `@this`: The `IServiceCollection` instance to extend. + +--- + +### 3. `AzureBlobStorageProvider` +Implements `IBlobStorageProvider` for Azure Blob Storage operations. + +#### Public Constructors +```csharp +public AzureBlobStorageProvider() +public AzureBlobStorageProvider(BlobServiceClient blobServiceClient) +public AzureBlobStorageProvider(BlobServiceClient blobServiceClient, string containerName) +``` +- **Purpose:** Initializes the provider. + - Default constructor uses the connection string from `AzureBlobInfo.StorageConnectionString`. + - Parameterized constructors allow a custom `BlobServiceClient` and optional container name. +- **Parameters:** + - `blobServiceClient`: An instance of `BlobServiceClient`. + - `containerName`: Optional custom container name (defaults to `AzureBlobInfo.StorageContainer`). + +##### `CostsToCheckExistence()` +```csharp +public bool CostsToCheckExistence() +``` +- **Purpose:** Indicates if checking file existence incurs a cost. +- **Returns:** `true` (Azure Blob operations have a cost). + +##### `SaveAsync(Blob document)` +```csharp +public async Task SaveAsync(Blob document) +``` +- **Purpose:** Uploads a `Blob` to Azure Blob Storage. +- **Parameters:** + - `document`: The `Blob` object to save. +- **Throws:** `Exception` with debug logging if the upload fails. + +##### `SaveAsync(Blob document, string key)` +```csharp +public async Task SaveAsync(Blob document, string key) +``` +- **Purpose:** Uploads a `Blob` to Azure Blob Storage with a specified key. +- **Parameters:** + - `document`: The `Blob` object to save. + - `key`: The custom key to use (defaults to `document.GetKey()` if null/empty). +- **Throws:** `Exception` with debug logging if the upload fails. + +##### `FileExistsAsync(Blob document)` +```csharp +public async Task FileExistsAsync(Blob document) +``` +- **Purpose:** Checks if a `Blob` exists in Azure Blob Storage. +- **Parameters:** + - `document`: The `Blob` object to check. +- **Returns:** `true` if the file exists, `false` otherwise. +- **Throws:** `Exception` with error logging if the check fails. + +##### `FileExistsAsync(Blob document, string key)` +```csharp +public async Task FileExistsAsync(Blob document, string key) +``` +- **Purpose:** Checks if a file exists in Azure Blob Storage with a specified key. +- **Parameters:** + - `document`: The `Blob` object (used if `key` is null/empty). + - `key`: The custom key to check (defaults to `document.GetKey()` if null/empty). +- **Returns:** `true` if the file exists, `false` otherwise. +- **Throws:** `Exception` with error logging if the check fails. + +##### `LoadAsync(Blob document)` +```csharp +public async Task LoadAsync(Blob document) +``` +- **Purpose:** Downloads a `Blob` from Azure Blob Storage as a byte array. +- **Parameters:** + - `document`: The `Blob` object to load. +- **Returns:** A byte array of the file content. +- **Throws:** `Exception` with error logging if loading fails. + +##### `DeleteAsync(Blob document)` +```csharp +public async Task DeleteAsync(Blob document) +``` +- **Purpose:** Deletes a `Blob` from Azure Blob Storage. +- **Parameters:** + - `document`: The `Blob` object to delete. +- **Behavior:** Does nothing if `document` is empty. +- **Throws:** `Exception` with error logging if deletion fails. + +##### `DeleteAsync(Blob document, string key)` +```csharp +public async Task DeleteAsync(Blob document, string key) +``` +- **Purpose:** Deletes a file from Azure Blob Storage with a specified key. +- **Parameters:** + - `document`: The `Blob` object (used if `key` is null/empty). + - `key`: The custom key to delete (defaults to `document.GetKey()` if null/empty). +- **Behavior:** Does nothing if `document` is empty. +- **Throws:** `Exception` with error logging if deletion fails. + +--- + +### 4. `AzureBlobInfo` +A utility class providing Azure Blob Storage configuration details. + +#### Public Properties +- `StorageAccountKey`: Gets the storage account key from configuration (read-only, type: `string`). + - **Source:** Retrieved from `Config.Get("AzureStorage:AccountKey")`. + +--- + +## Usage Example +```csharp +// Setup services +services.AddAzureBlobStorageProvider(); +var provider = Context.Current; + +// Get the storage provider +var storageProvider = provider.GetService(); + +// Upload a file +var blob = new Blob(File.ReadAllBytes("example.txt"), "example.txt"); +await storageProvider.SaveAsync(blob); + +// Check existence +var exists = await storageProvider.FileExistsAsync(blob); + +// Load content +var content = await storageProvider.LoadAsync(blob); + +// Generate SAS URI +var sasGenerator = new SASTokenGenerator(); +var sasUri = sasGenerator.GenerateContainerSasUri(); + +// Delete file +await storageProvider.DeleteAsync(blob); +``` \ No newline at end of file From 2194d89c8bd989ce68e473ed415844250dd0c0cb Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Fri, 28 Feb 2025 15:37:51 +0000 Subject: [PATCH 21/68] Olive.Cloud doc --- docs/Api/Olive.Cloud.md | 95 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 docs/Api/Olive.Cloud.md diff --git a/docs/Api/Olive.Cloud.md b/docs/Api/Olive.Cloud.md new file mode 100644 index 000000000..d9c1074ca --- /dev/null +++ b/docs/Api/Olive.Cloud.md @@ -0,0 +1,95 @@ +# Olive.Cloud + +## Overview +The `Olive.Cloud` is an abstract base class designed to manage and load secrets from a cloud-based provider. It stores secrets in a dictionary and updates the application configuration with the retrieved values. + +## Namespace +```csharp +namespace Olive.Cloud; +``` + +## Class: `Secrets` +### Description +This abstract class facilitates the retrieval and storage of secrets within an application. The secrets are loaded from a cloud-based provider and stored in memory for use during runtime. + +### Events +#### `Loaded` +- **Type:** `AwaitableEventHandler` +- **Description:** This event is triggered when the secrets are successfully loaded. + +### Fields +#### `Config` +- **Type:** `IConfiguration` +- **Access Modifier:** `protected` +- **Description:** Holds the application configuration object. + +#### `SecretString` +- **Type:** `string` +- **Access Modifier:** `protected` +- **Description:** Stores the downloaded secrets as a JSON string. + +### Properties +#### `SecretId` +- **Type:** `string` +- **Access Modifier:** `protected abstract` +- **Description:** Specifies the unique identifier for the secret in the cloud provider. + +### Methods +#### `DownloadSecrets()` +- **Access Modifier:** `protected abstract` +- **Return Type:** `string` +- **Description:** An abstract method that must be implemented in derived classes to download secrets from a cloud provider. + +#### `Load()` +- **Access Modifier:** `public` +- **Return Type:** `void` +- **Description:** Downloads and loads secrets into the application configuration. +- **Implementation Steps:** + 1. Calls `Download()` to retrieve the secret values. + 2. Parses the retrieved secrets as JSON. + 3. Updates the application configuration with the retrieved secret values. + 4. Raises the `Loaded` event. + +#### `Download()` +- **Access Modifier:** `private` +- **Return Type:** `void` +- **Description:** Downloads secrets and handles any errors that occur during retrieval. +- **Error Handling:** + - Logs errors when secrets cannot be retrieved. + - Throws exceptions when retrieval fails. +- **Implementation Steps:** + 1. Calls `DownloadSecrets()` to retrieve the secrets. + 2. Validates the retrieved data to ensure it is not empty. + 3. Stores the retrieved secrets in `SecretString`. + +### Logging +- Uses `ILogger` for logging errors and exceptions. +- Logs errors when secret retrieval fails. + +## Usage Example +Since `Secrets` is an abstract class, it must be inherited by a concrete class that implements `DownloadSecrets()`. Below is an example implementation: + +```csharp +public class MyCloudSecrets : Secrets +{ + protected override string SecretId => "MySecretKey"; + + public MyCloudSecrets(IConfiguration config) : base(config) {} + + protected override string DownloadSecrets() + { + // Replace this with actual cloud secret retrieval logic. + return "{ \"ApiKey\": \"12345\", \"DbPassword\": \"securepassword\" }"; + } +} +``` + +To use this class in an application: + +```csharp +IConfiguration configuration = new ConfigurationBuilder().Build(); +MyCloudSecrets secrets = new MyCloudSecrets(configuration); +secrets.Load(); +Console.WriteLine(configuration["ApiKey"]); // Output: 12345 +``` + From 2dd6761829eb13f56fc8ebd2efb49475f21e2c98 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Fri, 28 Feb 2025 16:02:13 +0000 Subject: [PATCH 22/68] Olive.Compression doc --- docs/Api/Olive.Compression.md | 56 +++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) create mode 100644 docs/Api/Olive.Compression.md diff --git a/docs/Api/Olive.Compression.md b/docs/Api/Olive.Compression.md new file mode 100644 index 000000000..e4580b2a3 --- /dev/null +++ b/docs/Api/Olive.Compression.md @@ -0,0 +1,56 @@ +**Olive.Compression** + +**Overview:** +The `Olive.Compression` provides methods for compressing and decompressing files and directories using different compression formats, such as ZIP, GZIP, and TAR. + +**Methods:** + +1. **Compress(IEnumerable, CompressionFormat)** + - Compresses a list of files into a specified compression format. + - Throws an exception if duplicate filenames exist. + - Returns the compressed data as a byte array. + +2. **Compress(DirectoryInfo, CompressionFormat)** + - Compresses an entire directory into a specified compression format. + - Returns the compressed data as a byte array. + +3. **Compress(DirectoryInfo, FileInfo, CompressionFormat, bool)** + - Creates a compressed file from a directory and saves it to a specified destination. + - Supports ZIP, GZIP, and TAR formats. + - Throws an exception if the destination file already exists unless overwrite is set to `true`. + +4. **Decompress(FileInfo, DirectoryInfo, bool, bool)** + - Decompresses a compressed file into a specified directory. + - Supports extracting full paths and overwriting existing files. + +--- + +**SevenZip Documentation** + +**Overview:** +The `SevenZip` class provides an interface to compress and decompress files using the 7-Zip utility. + +**Properties:** + +- `SEVEN_ZIP_EXE_FILE_PATH`: Specifies the path of the 7-Zip executable. + +**Methods:** + +1. **Compress(string, string[])** + - Compresses specified folders into a 7-Zip archive. + +2. **Compress(string, int?, string[])** + - Supports splitting archives into parts of a specified size (in KB). + +3. **Compress(string, int?, CompressionMode, string, string[], string[])** + - Compresses folders into a 7-Zip archive with custom parameters and optional exclusion filters. + +4. **Compress(IEnumerable, CompressionMode, string)** + - Compresses a list of files into a temporary 7-Zip file and returns the resulting file. + +5. **CompressToBytes(IEnumerable, CompressionMode, string)** + - Compresses a list of files into a 7-Zip archive and returns the compressed data as a byte array. + +6. **Compress(FileInfo, IEnumerable, CompressionMode, string)** + - Compresses specified files into a 7-Zip archive and saves it to a specified destination. + From 134ff8faabf9e44c47528e5cc70f724c27621680 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Mon, 3 Mar 2025 07:50:44 +0000 Subject: [PATCH 23/68] Olive.Console doc --- docs/Core/Olive.Console.md | 95 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 docs/Core/Olive.Console.md diff --git a/docs/Core/Olive.Console.md b/docs/Core/Olive.Console.md new file mode 100644 index 000000000..a574117fd --- /dev/null +++ b/docs/Core/Olive.Console.md @@ -0,0 +1,95 @@ +# Olive.Console + +## Overview +Olive is used by default in M# project templates. You can also add Olive nuget packages to any other project to use the extension methods. + +Some of the Olive features however, will need the core Olive context and services to be set up and available before you can use them. +This includes EventBus and various other specific Olive services. + +The `Application` class is a host container designed for console applications that leverage Olive services. It provides an entry point for initializing and running console-based applications with dependency injection, logging, and configuration management. + +## Methods + +### `Start(string[] args, Action configure = null)` +- **Description**: Initializes and runs the console application asynchronously. +- **Parameters**: + - `args`: Command-line arguments passed to the application. + - `configure`: An optional configuration action for the `HostBuilder`. +- **Returns**: `Task` representing the asynchronous operation. +- **Usage**: + ```csharp + await Application.Start(args); + ``` + +### `StartUnitTest(Action configure = null)` +- **Description**: Initializes the application in unit test mode. +- **Parameters**: + - `configure`: An optional configuration action for `HostBuilder`. +- **Usage**: + ```csharp + Application.StartUnitTest(); + ``` + +--- + +# Startup Class Documentation + +## Overview +The `Startup` class provides a base implementation for configuring services and running console applications in an Olive-based system. + +## Methods + +### `ConfigureServices(IServiceCollection services)` +- **Description**: Configures required services for the application. +- **Usage**: + ```csharp + public override void ConfigureServices(IServiceCollection services) { + services.AddLogging(); + } + ``` + +--- + +## Example Usage +To activate the Olive context in a console app add a nuget reference to `Olive`, `Olive.Console` and any other required Olive services. Then replace your Program startup file with the following: + +```csharp +class Program +{ + static Task Main(string[] args) => Olive.Console.Application.Start(args); +} + +class Startup : Olive.Console.Startup +{ + public Startup(IConfiguration config) : base(config) + { + // Using AWS services? + // config.LoadAwsDevIdentity("...", "...", Amazon.RegionEndpoint.EUWest1, loadSecrets: false); + } + + public override void ConfigureServices(IServiceCollection services) + { + // ... any specific configurations will go here. + + services.AddDefaultAudit(); + + // Using AWS stuff?? + services.AddAwsEventBus(); + services.AddAWSService(); + + // Using database? + services.AddDatabase(); + services.AddDataAccess(x => x.SqlServer()); + } + + static IDatabase Database => Context.Current.Database(); + + protected override Task Run() + { + Database.ConfigDataAccess().Configure(); + + Console.WriteLine("Hello!"); + return Task.CompletedTask; + } +} +``` From c09c7299ec8588c120ae87dc82842857586acbee Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Mon, 3 Mar 2025 08:41:56 +0000 Subject: [PATCH 24/68] Olive.Dashboards.PowerBI doc --- .../Olive.Dashboards.PowerBI.md | 39 +++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100644 docs/Olive.Dashboards.PowerBI/Olive.Dashboards.PowerBI.md diff --git a/docs/Olive.Dashboards.PowerBI/Olive.Dashboards.PowerBI.md b/docs/Olive.Dashboards.PowerBI/Olive.Dashboards.PowerBI.md new file mode 100644 index 000000000..dc1774c17 --- /dev/null +++ b/docs/Olive.Dashboards.PowerBI/Olive.Dashboards.PowerBI.md @@ -0,0 +1,39 @@ +**Olive.Dashboards.PowerBI** +This documentation provides an overview of the Power BI embedding service and Azure AD authentication for accessing Power BI resources. + +**Class:** `AzureAdService` + +**Description:** +This class is responsible for generating an Azure Active Directory (AAD) authentication token for Power BI services. It supports both `masteruser` and `serviceprincipal` authentication modes. + +**Methods:** +- `Task GetAccessToken()`: Retrieves an AAD access token using either `masteruser` credentials or a `service principal`. + +**Usage:** +- Call `GetAccessToken()` to obtain an access token that can be used to authenticate API requests to Power BI. + +--- + +**Embed Service Documentation** + +**Class:** `EmbedService` + +**Description:** +This class provides functionality to embed Power BI reports and dashboards using Azure authentication. + +**Constructor:** +- `EmbedService(AzureAdService azureAdService, string uri = "https://api.powerbi.com")` + +**Methods:** +- `Task EmbedReport(Guid workspaceId, Guid reportId, EmbedToken embedToken)`: Embeds a report using an existing embed token. +- `Task EmbedReport(Guid workspaceId, Guid reportId, [Optional] Guid additionalDatasetId)`: Embeds a report and generates an embed token. +- `Task GetEmbedTokenForReport(Guid workspaceId, IList datasetIds, Guid reportId)`: Retrieves an embed token for a report. +- `Task RefreshDatasetAsync(Guid workspaceId, string datasetId, DatasetRefreshRequest datasetRefreshRequest = null)`: Refreshes a Power BI dataset. +- `Task> GetDatasetRefreshHistoryAsync(Guid workspaceId, string datasetId)`: Retrieves the refresh history of a dataset. +- `Task EmbedDashboard(Guid workspaceId, Guid dashboardId, TokenAccessLevel tokenAccessLevel)`: Embeds a Power BI dashboard. +- `Task GetEmbedTokenForDashboard(Guid workspaceId, Guid dashboardId, TokenAccessLevel tokenAccessLevel, string datasetId = null)`: Retrieves an embed token for a dashboard. + +**Usage:** +- Initialize `EmbedService` with an instance of `AzureAdService`. +- Use `EmbedReport` or `EmbedDashboard` to obtain the necessary embed details. +- Use `RefreshDatasetAsync` to refresh datasets as needed. \ No newline at end of file From d8e23b62f1eeef1737405da3c4929891b1c5bc11 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Mon, 3 Mar 2025 09:25:03 +0000 Subject: [PATCH 25/68] Olive.Drawing doc --- docs/Olive.Drawing/Olive.Drawing.md | 86 +++++++++++++++++++++++++++++ 1 file changed, 86 insertions(+) create mode 100644 docs/Olive.Drawing/Olive.Drawing.md diff --git a/docs/Olive.Drawing/Olive.Drawing.md b/docs/Olive.Drawing/Olive.Drawing.md new file mode 100644 index 000000000..f4c467cc3 --- /dev/null +++ b/docs/Olive.Drawing/Olive.Drawing.md @@ -0,0 +1,86 @@ +**Olive.Drawing** + +### Overview +The `Olive.Drawing` namespace provides a set of utilities for image processing, including optimization, resizing, rotation, compression, and color manipulation. These utilities leverage `SkiaSharp` and `System.Drawing` to offer efficient and high-quality image transformations. + +--- + +## **ImageOptimizer** + +### **Description** +The `ImageOptimizer` class is used to resize and optimize images while maintaining a balance between quality and file size. + +### **Properties** +- **MaximumWidth** (int): The maximum allowed width for the optimized image (default is 900). +- **MaximumHeight** (int): The maximum allowed height for the optimized image (default is 700). +- **Quality** (int): The quality setting for the output image (default is 80). +- **OutputFormat** (enum): The format of the output image (JPEG, PNG, BMP, GIF). + +### **Methods** +- **Optimize(SKBitmap source)**: Resizes and optimizes the input image while maintaining the aspect ratio. +- **Optimize(byte[] sourceData, bool toJpeg = true)**: Optimizes the given image byte array and returns a new byte array. +- **Optimize(string sourceImagePath, string optimizedImagePath)**: Reads an image from a file, optimizes it, and saves the result to a new file. + +--- + +## **GraphicExtensions** + +### **Description** +This static class contains helper methods for working with `Bitmap` images, including column extraction, rotation, and insertion of images. + +### **Methods** +- **GetColumn(Bitmap image, int columnIndex)**: Extracts a single-column image from the specified bitmap. +- **GetWidth(Font font, string text, bool useAntialias)**: Calculates the width of a given text using the specified font. +- **Rotate(Image image, double degrees)**: Rotates an image clockwise by the specified number of degrees. +- **Stretch(Bitmap image, int width)**: Stretches a single-column image to a new width. +- **SaveAsGif(Bitmap image, string path, bool transparent)**: Saves an image as a GIF with optional transparency. + +--- + +## **GifPalleteGenerator** + +### **Description** +This class generates a color palette for GIF images based on the colors used in the image. + +### **Methods** +- **GeneratePallete(Bitmap image)**: Creates a color palette optimized for the given image. +- **FindAllColours(Bitmap image)**: Identifies and sorts all unique colors in an image by frequency. + +--- + +## **BitmapHelper** + +### **Description** +Provides various helper functions for working with bitmap images, including resizing, brightness adjustment, and cropping. + +### **Methods** +- **ToBuffer(Image image, ImageFormat format, int quality = 100)**: Converts an image to a byte array with the specified format and quality. +- **FromBuffer(byte[] buffer)**: Converts a byte array to an image. +- **IsValidImage(byte[] buffer)**: Checks if the given byte array contains a valid image. +- **Resize(Image source, int newWidth, int newHeight)**: Resizes an image to the specified dimensions. +- **Brighten(Image source, int level)**: Adjusts the brightness of an image. +- **Crop(Image image, Rectangle rectangle)**: Crops an image to the specified rectangle. + +--- + +## **Extensions (Blob Optimization)** + +### **Description** +These methods provide image optimization capabilities for `Blob` objects, commonly used in storage and database applications. + +### **Methods** +- **OptimizeImage(Blob blob, int maxWidth, int maxHeight, int quality, bool toJpeg = true)**: Optimizes an image stored as a `Blob`. +- **SetData(byte[] data)**: Updates the `Blob` with optimized image data. + +--- + +## **Use Cases** +- **Web Applications:** Optimize images before serving them to users to reduce bandwidth usage. +- **Database Storage:** Store images efficiently with controlled dimensions and quality. +- **Image Processing Pipelines:** Apply transformations such as cropping, resizing, and brightness adjustments before analysis. +- **GIF Optimization:** Generate optimized GIFs with reduced color palettes to save space. + +--- + +This documentation provides an overview of the key functionalities in `Olive.Drawing`. For further details, refer to the implementation or usage examples in the source code. + From 74333fa1aa82c17923daace6b23288bd9997051d Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Mon, 3 Mar 2025 10:16:24 +0000 Subject: [PATCH 26/68] Olive.Email.Imap doc --- docs/Services/Olive.Email.Imap.md | 130 ++++++++++++++++++++++++++++++ 1 file changed, 130 insertions(+) create mode 100644 docs/Services/Olive.Email.Imap.md diff --git a/docs/Services/Olive.Email.Imap.md b/docs/Services/Olive.Email.Imap.md new file mode 100644 index 000000000..ca197e3c8 --- /dev/null +++ b/docs/Services/Olive.Email.Imap.md @@ -0,0 +1,130 @@ +# Olive.Email.Imap + +## Overview +This documentation provides details about the `Olive.Email.Imap` and `ImapService`, which are used for handling email failures and interacting with an IMAP server to retrieve and process emails. + +--- + +## **BaseEmailFailureService** +### **Description** +The `BaseEmailFailureService` is an abstract class that handles failure emails by fetching new messages from an IMAP service, checking if they indicate a failure, and marking the corresponding email messages as failed in the database. + +### **Constructor** +```csharp +public BaseEmailFailureService( + IImapService imapService, + IDatabase database, + ILogger logger) +``` +- `imapService`: An instance of `IImapService` to retrieve emails. +- `database`: Database access for storing email failure logs. +- `logger`: Logging service for recording email failure events. + +### **Methods** +#### **Check()** +```csharp +public async Task Check() +``` +- Retrieves new email messages. +- Checks if an email is a failure. +- Marks related emails as failure in the database. +- Marks emails as seen in IMAP. + +--- + +## **IImapService** +### **Interface Definition** +```csharp +public interface IImapService +``` +Defines a contract for IMAP-based email retrieval and processing. + +### **Methods** +#### **GetNewMessage(string folder = "Inbox")** +```csharp +Task> GetNewMessage(string folder = "Inbox"); +``` +- Retrieves new messages from a specified folder. + +#### **GetReferences(MimeMessage main)** +```csharp +Task> GetReferences(MimeMessage main); +``` +- Fetches referenced messages linked to the given email. + +#### **MarkAsSeen(MimeMessage message)** +```csharp +Task MarkAsSeen(MimeMessage message); +``` +- Marks an email as read. + +--- + +## **ImapService Implementation** +### **Class Definition** +```csharp +public class ImapService : IImapService +``` +Provides an implementation for the IMAP service using `MailKit` to interact with email servers. + +### **Constructor** +```csharp +public ImapService(IConfiguration configuration) +``` +- `configuration`: Retrieves email server settings. + +### **Methods** +#### **GetNewMessage(string folder = "Inbox")** +```csharp +public async Task> GetNewMessage(string folder = "Inbox") +``` +- Connects to IMAP. +- Searches for unread messages. +- Retrieves new messages. + +#### **GetReferences(MimeMessage main)** +```csharp +public async Task> GetReferences(MimeMessage main) +``` +- Searches for emails that reference the given message (i.e., email threads). + +#### **MarkAsSeen(MimeMessage message)** +```csharp +public async Task MarkAsSeen(MimeMessage message) +``` +- Marks a message as read using IMAP flags. + +--- + +## **Dependency Injection Extensions** +### **AddEmailFailureService()** +```csharp +public static IServiceCollection AddEmailFailureService(this IServiceCollection services) +``` +- Registers an email failure handling service. + +### **AddImapService()** +```csharp +public static IServiceCollection AddImapService(this IServiceCollection services) +``` +- Registers the `ImapService`. + +--- + +## **Usage Example** +```csharp +services.AddEmailFailureService(); +services.AddImapService(); + +var provider = Context.Current; +var emailFailureService = provider.GetRequiredService(); +await emailFailureService.Check(); +``` +- Registers and initializes the email failure service. +- Calls `Check()` to process new emails. + +--- + +## **Conclusion** +This system efficiently processes failure emails and interacts with an IMAP server to retrieve and mark emails. The modular design ensures that services can be extended and customized for different email processing needs. + From 1d09c04c1fc20f3adb99a3e2c0309432fabef17b Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Mon, 3 Mar 2025 11:41:37 +0000 Subject: [PATCH 27/68] Olive.Encryption doc --- docs/Core/Utilities/Olive.Encryption.md | 271 ++++++++++++++++++++++++ 1 file changed, 271 insertions(+) create mode 100644 docs/Core/Utilities/Olive.Encryption.md diff --git a/docs/Core/Utilities/Olive.Encryption.md b/docs/Core/Utilities/Olive.Encryption.md new file mode 100644 index 000000000..fc21a41bc --- /dev/null +++ b/docs/Core/Utilities/Olive.Encryption.md @@ -0,0 +1,271 @@ +# Olive.Encryption + +## Overview + +This document provides details about the encryption, decryption, hashing, and key management utilities available in the `Olive.Security` namespace. The utilities facilitate symmetric encryption (AES), asymmetric encryption (RSA), and secure password hashing using the PBKDF2 algorithm. + +--- + +## Table of Contents +- [Symmetric Encryption (AES)](#symmetric-encryption-aes) +- [Asymmetric Encryption (RSA)](#asymmetric-encryption-rsa) +- [Secure Password Hashing (PBKDF2)](#secure-password-hashing-pbkdf2) +- [Utility Classes and Extensions](#utility-classes-and-extensions) +- [Examples](#examples) + +--- + +## Symmetric Encryption (AES) + +This utility enables encrypting and decrypting data symmetrically based on the AES algorithm. + +### Methods: + +#### Encryption.Encrypt() + +Encrypts the specified text or binary data using AES encryption. + +Overloads: + +- Encrypt string to Base64: + ```csharp + string Encrypt(string raw, string password, Encoding encoding = null) + ``` + +- Encrypt byte array to byte array: + ```csharp + byte[] Encrypt(byte[] raw, string password) + ``` + +**Example usage:** +```csharp +var plaintext = "Hello World!"; +var password = "SecurePassword123"; + +string encrypted = Encryption.Encrypt(plaintext, password); +// encrypted now contains the Base64 AES encrypted data. +``` + +--- + +#### Encryption.Decrypt() + +Decrypts AES-encrypted data back to original plaintext. + +Overloads: + +- Decrypt Base64 string to plaintext: + ```csharp + string Decrypt(string cipher, string password, Encoding encoding = null) + ``` + +- Decrypt byte array to original byte array: + ```csharp + byte[] Decrypt(byte[] cipher, string password) + ``` + +**Example usage:** +```csharp +var decryptedText = Encryption.Decrypt(encrypted, password); +// decryptedText will return original string "Hello World!". +``` + +--- + +## Asymmetric Encryption (RSA) + +Provides RSA asymmetric cryptography methods, allowing to encrypt data with a public key and decrypt with the corresponding private key. + +### Class: Encryption.AsymmetricKeyPair + +The `AsymmetricKeyPair` class generates and holds RSA cryptographic keys: + +| Property | Description | +| -------- | ----------- | +| `EncryptionKey` | The RSA public key (Base64 string). Use this key to encrypt data. | +| `DecryptionKey` | The RSA private key (Base64 string). Use this to decrypt previously encrypted data. | + +### Methods: + +#### AsymmetricKeyPair.Generate() + +Generates a new RSA public/private key pair. + +```csharp +AsymmetricKeyPair keyPair = Encryption.AsymmetricKeyPair.Generate(); +``` + +### RSA Encryption Methods: + +- **EncryptAsymmetric** + Encrypt byte array with RSA public key: + + ```csharp + byte[] EncryptAsymmetric(byte[] raw, string encryptKey) + ``` + +- **DecryptAsymmetric** + Decrypt byte array with RSA private key: + + ```csharp + byte[] DecryptAsymmetric(byte[] cipher, string decryptKey) + ``` + +**Example usage:** +```csharp +// Generate keys +var keyPair = Encryption.AsymmetricKeyPair.Generate(); + +byte[] data = Encoding.UTF8.GetBytes("Sensitive Information"); + +// Encrypt using public key +byte[] encryptedData = Encryption.EncryptAsymmetric(data, keyPair.EncryptionKey); + +// Decrypt using private key +byte[] originalData = Encryption.DecryptAsymmetric(encryptedData, keyPair.DecryptionKey); + +string originalText = Encoding.UTF8.GetString(originalData); // "Sensitive Information" +``` + +--- + +## Secure Password Hashing (PBKDF2) + +Implements secure password hashing using the PBKDF2 algorithm with salts and iterations. + +### Class: SecurePassword + +| Property | Description | +| -------- | ----------- | +| `Password` | Base64-encoded hashed password | +| `Salt` | Base64-encoded cryptographic salt | + +### Methods: + +- **SecurePassword.Create** + +Creates a salted PBKDF2 hash from a plaintext password. + +```csharp +SecurePassword Create(string password) +``` + +- **SecurePassword.Verify** + +Verifies a plaintext password against hashed password and salt. + +```csharp +bool Verify(string clearTextPassword, string hashedPassword, string salt) +``` + +**Example usage:** +```csharp +// Create hashed password: +var securedPass = SecurePassword.Create("MySecurePassword!"); + +// Verify Password: +bool isPasswordValid = SecurePassword.Verify("MySecurePassword!", securedPass.Password, securedPass.Salt); +``` + +--- + +## Utility Classes and Extensions + +### Class: EncryptionExtensions + +This class provides extension methods for RSA key serialization: + +- **ToKey** +Serializes RSAParameters as a Base64 string. + + ```csharp + string ToKey(this RSAParameters rsaParams) + ``` + +- **FromKey** +Deserializes RSAParameters from a Base64 string. + + ```csharp + RSACryptoServiceProvider FromKey(this RSACryptoServiceProvider rsaProvider, string key) + ``` + +**Example usage:** +```csharp +var rsa = new RSACryptoServiceProvider(); +var rsaParams = rsa.ExportParameters(false); + +// Serialize parameters to Base64 string: +string keyString = rsaParams.ToKey(); + +// Deserialize parameters from string: +rsa = rsa.FromKey(keyString); +``` + +--- + +## Examples: + +### AES encryption and decryption example: +```csharp +var originalText = "Confidential data"; +var password = "SuperSecret123"; + +// Encrypt +var encrypted = Encryption.Encrypt(originalText, password); + +// Decrypt +var decrypted = Encryption.Decrypt(encrypted, password); + +// Result: +// decrypted == "Confidential data" +``` + +### RSA asymmetric encryption and decryption example: +```csharp +// Generate key pair: +var keyPair = Encryption.AsymmetricKeyPair.Generate(); +var publicKey = keyPair.EncryptionKey; +var privateKey = keyPair.DecryptionKey; + +// Original data: +var originalBytes = Encoding.UTF8.GetBytes("Important data"); + +// Encryption: +var encryptedData = Encryption.EncryptAsymmetric(originalBytes, publicKey); + +// Decryption: +var decryptedData = Encryption.DecryptAsymmetric(encryptedData, privateKey); +var decryptedText = Encoding.UTF8.GetString(decryptedData); + +// Result: +// decryptedText == "Important data" +``` + +### SecurePassword hashing and verification example: +```csharp +// Hash password: +var securePwd = SecurePassword.Create("MyP@ssword!"); + +// Verify: +var isValid = SecurePassword.Verify("MyP@ssword!", securePwd.Password, securePwd.Salt); + +// Result: +// isValid == true +``` + +--- + +## Notes: +- Default encodings are UTF-8 if no encoding is specified. +- AES parameters default to `AES_KEY_LENGTH = 32 bytes` and `AES_IV_LENGTH = 16 bytes`. + +--- + +## Exception Handling: +Methods throw `ArgumentNullException` if required parameters are null or empty. For instance: +```csharp +// Throws ArgumentNullException because 'raw' is empty +string encrypted = Encryption.Encrypt("", "password"); +``` + +Please handle exceptions properly in your applications. \ No newline at end of file From 34f916af8aa5dec9cd8c4229b6506ae8d75e4c7e Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Mon, 3 Mar 2025 12:01:46 +0000 Subject: [PATCH 28/68] Olive.Entities.Cache.Redis doc --- docs/Entities/Olive.Entities.Cache.Redis.md | 234 ++++++++++++++++++++ 1 file changed, 234 insertions(+) create mode 100644 docs/Entities/Olive.Entities.Cache.Redis.md diff --git a/docs/Entities/Olive.Entities.Cache.Redis.md b/docs/Entities/Olive.Entities.Cache.Redis.md new file mode 100644 index 000000000..57dabef36 --- /dev/null +++ b/docs/Entities/Olive.Entities.Cache.Redis.md @@ -0,0 +1,234 @@ +# Olive.Entities.Cache.Redis + +## Overview + +`RedisCacheProvider` is a caching implementation utilizing Redis for efficient caching of Olive.Entities (entities) data objects. It supports storing, retrieving, and managing cached entries including individual entities and entity collections. This provider integrates seamlessly with the Olive caching infrastructure through dependency injection. + +--- + +## Table of Contents + +- [RedisCacheProvider](#rediscacheprovider) + - [Constructor](#constructor) + - [Methods](#methods) +- [RedisCacheExtensions](#rediscacheextensions) +- [Examples](#examples) + +--- + +## RedisCacheProvider + +### Constructor + +Initializes the Redis connection using the provided Redis configuration string from settings (`Database:Cache:RedisConfig`). By default, it connects to `localhost:6379`. + +**Usage:** +```csharp +var provider = new RedisCacheProvider(); +``` + +### Methods + +#### Add(IEntity entity) + +Stores a single entity instance in Redis cache. + +- **Parameters** + - `entity`: An instance of IEntity to cache. + +**Example:** +```csharp +provider.Add(myEntity); +``` + +--- + +#### Get(Type entityType, string id) + +Retrieves a single entity instance from Redis cache. + +- **Parameters** + - `entityType`: Type of the entity to retrieve. + - `id`: The unique identifier of the entity. + +- **Returns:** The cached entity instance, or null if not found. + +**Example:** +```csharp +var cachedEntity = provider.Get(typeof(User), "user-123"); +``` + +--- + +#### Remove(IEntity entity) + +Removes a cached entity instance from Redis. + +- **Parameters** + - `entity`: The entity instance to remove. + +**Example:** +```csharp +provider.Remove(myEntity); +``` + +--- + +#### AddList(Type type, IEnumerable list) + +Caches a list or collection of entities in Redis. + +- **Parameters** + - `type`: The type of entities stored. + - `list`: An IEnumerable containing the entity instances. + +**Example:** +```csharp +provider.AddList(typeof(User), usersCollection); +``` + +--- + +#### GetList(Type type) + +Retrieves a cached list of entities from Redis cache. + +- **Parameters** + - `type`: The type of entities to retrieve. + +- **Returns:** IEnumerable containing the entity instances, or null if not found. + +**Example:** +```csharp +var users = provider.GetList(typeof(User)); +``` + +--- + +#### RemoveList(Type type) + +Removes a cached list of entities based on entity type. + +- **Parameters** + - `type`: The type of entities whose cache is to be cleared. + +**Example:** +```csharp +provider.RemoveList(typeof(User)); +``` + +--- + +#### ClearAll() + +Flushes the entire Redis database, removing all keys and values. + +**Warning:** This action clears all cached data. Use with caution. + +**Example:** +```csharp +provider.ClearAll(); +``` + +--- + +#### Remove(Type type, bool invalidateCachedReferences = false) + +Removes cached entities by type. Can optionally invalidate cached references. + +- **Parameters** + - `type`: The entity type whose cached instances should be removed. + - `invalidateCachedReferences`: (Currently unused and set as default: false) + +**Example:** +```csharp +provider.Remove(typeof(User)); +``` + +--- + +## RedisCacheExtensions + +### AddRedisCache(this IServiceCollection services) + +An extension method to easily register `RedisCacheProvider` as the application's caching provider via dependency injection, using singleton lifetime. + +**Example Usage in Startup.cs:** +```csharp +public void ConfigureServices(IServiceCollection services) +{ + services.AddRedisCache(); +} +``` + +--- + +## Examples + +### Example 1: Adding and retrieving an entity to/from Redis cache + +```csharp +var redisProvider = new RedisCacheProvider(); + +// Add entity to cache +redisProvider.Add(userEntity); + +// Retrieve entity from cache +var cachedUser = redisProvider.Get(typeof(User), userEntity.ID.ToString()) as User; +``` + +--- + +### Example 2: Caching a list of entities + +```csharp +var redisProvider = new RedisCacheProvider(); +var userList = new List { user1, user2, user3 }; + +// Cache list +redisProvider.AddList(typeof(User), userList); + +// Get cached list +var cachedUsers = redisProvider.GetList(typeof(User)) as IEnumerable; +``` + +--- + +### Example 3: Registering RedisCacheProvider in Startup.cs + +```csharp +public class Startup +{ + public void ConfigureServices(IServiceCollection services) + { + // Add Redis caching provider + services.AddRedisCache(); + + // Other configurations... + } +} +``` + +--- + +## Notes: + +- Internally, objects are serialized using binary formatter. Ensure types are serializable when cached. +- Redis server configuration can be provided through application's configuration file under the key: + +```json +{ + "Database": { + "Cache": { + "RedisConfig": "127.0.0.1:6379" + } + } +} +``` + +- Default Redis configuration: `localhost:6379` +- Redis connection is initialized with `allowAdmin=true`, which enables administration commands like flushing database. + +**Special Consideration (Web Farms):** + +- The row version functionality (`IsUpdatedSince`, `UpdateRowVersion`) isn't supported in RedisCacheProvider and will return default or no operation. \ No newline at end of file From 6c325d7dd6b716725fcfc5229b9ce32929f8083e Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Mon, 3 Mar 2025 12:35:48 +0000 Subject: [PATCH 29/68] Olive.Entities.Data.DynamoDB doc --- docs/Entities/Olive.Entities.Data.DynamoDB.md | 177 ++++++++++++++++++ 1 file changed, 177 insertions(+) create mode 100644 docs/Entities/Olive.Entities.Data.DynamoDB.md diff --git a/docs/Entities/Olive.Entities.Data.DynamoDB.md b/docs/Entities/Olive.Entities.Data.DynamoDB.md new file mode 100644 index 000000000..85728e805 --- /dev/null +++ b/docs/Entities/Olive.Entities.Data.DynamoDB.md @@ -0,0 +1,177 @@ +# Olive.Entities.Data.DynamoDB + +## Overview + +The provided `Olive.Entities.Data.DynamoDB` library enables easy integration and usage of Amazon DynamoDB within Olive applications. It provides abstraction layers through convenient classes and methods to interact with DynamoDB, making CRUD operations efficient and straightforward. + +--- + +## Table of Contents +- [Key Components](#key-components) +- [Dynamo](#dynamo-class) +- [DynamoDBDataProvider\](#dynamodbdataprovidert-class) +- [DynamoDBDataProviderFactory](#dynamodbdataproviderfactory-class) +- [DynamoRegistrationExtensions](#dynamoregistrationextensions-class) +- [Usage Examples](#usage-examples) + +--- + +## Key Components + +- **Dynamo**: Provides quick access to DynamoDB Context and Client for general purpose data operations. +- **DynamoDBDataProvider**: Generic data provider for managing entities within DynamoDB. +- **DynamoDBDataProviderFactory**: Factory class for generating data providers for entity types. +- **DynamoRegistrationExtensions**: Registers DynamoDB services within .NET Core Dependency Injection system. + +--- + +## Dynamo Class + +### Static Properties: + +| Property | Description | +|----------|-------------| +| `Client` | Provides the `IAmazonDynamoDB` client and accesses the underlying AWS DynamoDB service. | +| `Db` | Provides DynamoDB Context (`IDynamoDBContext`) for performing database operations more conveniently.| + +### Nested Classes: + +1. `DbIndex`: Access DynamoDB secondary indexes easily. + - Methods: + - `FirstOrDefault(object hashKey)`: Returns first entity or default for the provided hash key. + - `All(object hashKey)`: Returns an array of all entities matching the hash key. + +2. `DbTable`: Access standard DynamoDB tables operations. + - Methods: + - `Get(object obj)`: Retrieves an entity by primary key. + - `All(params ScanCondition[] conditions)`: Returns entities based on specified conditions. + - `ForAll(...)`: Performs a parallel action on all entities matching scan conditions. + +--- + +## DynamoDBDataProvider\ Class + +Implements Olive's `IDataProvider` interface to connect and manipulate entities in DynamoDB easily. + +### Important Methods: + +| Method | Description | +|--------|-------------| +| `Get(object objectID)` | Retrieves a single entity by its primary key. | +| `GetList(IDatabaseQuery query)` | Returns entities based on specified Olive queries (including hash key, index, or general conditions). | +| `Save(IEntity record)` | Saves changes (inserts new or updates existing entity). | +| `Delete(IEntity record)` | Deletes specified entity. | +| `BulkInsert(...)` | Inserts multiple entities in batch. | +| `BulkUpdate(...)` | Updates multiple entities in a batch (by deleting old and inserting updated versions). | + +### Notes on Implementation: + +- DynamoDB attributes like `[DynamoDBHashKey]` and `[DynamoDBGlobalSecondaryIndexHashKey]` are leveraged for efficient querying. +- Update operations are optimized by leveraging DynamoDB's capability to perform attribute updates. + +--- + +## DynamoDBDataProviderFactory Class + +This factory class generates and caches `DynamoDBDataProvider` instances, improving performance by reusing existing providers. + +### Methods: +- `GetProvider(Type type)`: Obtains or creates a cached provider for the specified entity type. +- `SupportsPolymorphism()`: Returns true, indicating DynamoDB supports entity polymorphism. + +--- + +## DynamoRegistrationExtensions Class + +This static class provides extension methods for easy registration of DynamoDB services using Microsoft's Dependency Injection pattern. + +### Extension Method: +- `.AddDynamoDB(IServiceCollection services, IConfiguration config)`: Adds DynamoDB support to your .NET application startup configuration. + +**Example usage (typically in Startup.cs):** +```csharp +public void ConfigureServices(IServiceCollection services) +{ + services.AddDynamoDB(Configuration); +} +``` + +--- + +## Usage Examples + +### Example 1: Getting an entity from DynamoDB + +```csharp +var provider = new DynamoDBDataProvider(); +var user = await provider.Get(userId); +``` + +### Example 2: Querying entities using hash key index + +```csharp +var index = Dynamo.Index("UserEmailIndex"); +var matchingUser = await index.FirstOrDefault("test@example.com"); +``` + +### Example 3: Saving an entity + +```csharp +var provider = new DynamoDBDataProvider(); +var user = new User { Id = "123", Name = "John Doe" }; +await provider.Save(user); +``` + +### Example 4: Bulk insert entities + +```csharp +var provider = new DynamoDBDataProvider(); +var users = new IEntity[] { user1, user2, user3 }; +await provider.BulkInsert(users, batchSize: 25); +``` + +### Example 5: Setting up DynamoDB in your .NET application + +```csharp +services.AddDynamoDB(Configuration); +// other configurations... +``` + +--- + +## Special Notes: + +- DynamoDB caching is implemented within `DynamoDBDataProviderFactory` to optimize performance. +- Changes in entities are tracked and finalized by methods like `OnEntityLoaded()`. +- Understand DynamoDB's data model (HashKey, SortKey, Indexes) to create efficient queries and schemas. + +--- + +## Exception and Not Implemented Notices: + +Certain methods (`Aggregate`, `ExecuteNonQuery`, etc.) directly inherited from Olive's `IDataProvider` are intentionally not implemented (`throw new NotImplementedException();`), as DynamoDB doesn't intrinsically support all relational database functionalities. + +**Caution:** Use implemented methods compatible with DynamoDB operations. + +--- + +## Dependencies + +These utilities require AWS DynamoDB SDK (`Amazon.DynamoDBv2`) and related packages (`Amazon.DynamoDBv2.DataModel`) to be installed and configured. + +--- + +## Configuration + +Ensure configuration keys in your `appsettings.json` (or equivalent) contains AWS credentials and endpoints if required: + +```json +{ + "AWS": { + "Profile": "default", + "Region": "us-west-2" + } +} +``` + +Alternatively, when using LocalStack for local AWS testing, ensure the correct LocalStack endpoint is configured. \ No newline at end of file From eeece9e6bd64c5c087914d425f2521d8520689f4 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Mon, 3 Mar 2025 13:15:20 +0000 Subject: [PATCH 30/68] Olive.Entities.Data.EF.Replication doc --- .../Olive.Entities.Data.EF.Replication.md | 227 ++++++++++++++++++ 1 file changed, 227 insertions(+) create mode 100644 docs/Entities/Olive.Entities.Data.EF.Replication.md diff --git a/docs/Entities/Olive.Entities.Data.EF.Replication.md b/docs/Entities/Olive.Entities.Data.EF.Replication.md new file mode 100644 index 000000000..71a478d26 --- /dev/null +++ b/docs/Entities/Olive.Entities.Data.EF.Replication.md @@ -0,0 +1,227 @@ +# Olive.Entities.Data.EF.Replication + +## Overview + +The Olive Entities EF Replication system simplifies database replication and synchronization across multiple data stores and services. It leverages Entity Framework (EF) and the Olive framework to provide flexible mechanisms to replicate, sync, and expose data via event-driven replication queues, enabling effective management of data movement across different system components. + +--- + +## Table of Contents + +- [Key Components](#key-components) +- [ExposedType Classes](#exposedtype-classes) + - [ExposedType](#exposedtypetdomain-class) + - [HardDeletableExposedType](#harddeletableexposedtype-class) + - [NakedExposedType & HardDeletableNakedExposedType](#nakedexposedtype--harddeletablenakedexposedtype-classes) +- [ExposedField Classes](#exposedfield-classes) +- [Replication Attributes](#replication-attributes) +- [Source and Destination Endpoints](#source-and-destination-endpoints) +- [Replication Messages](#replication-messages) +- [Examples](#examples) +- [Configuration](#configuration) + +--- + +## Key Components + +- **Exposed Types**: Define and expose domain models and their fields for replication. +- **Endpoints**: SourceEndpoint and DestinationEndpoint manage sending and receiving replicated data. +- **Replication Messages**: Structured messages (`ReplicateDataMessage`) for data synchronization. +- **Attributes**: Customize behavior through declarative attributes like `[ExportData]` and `[EventBus]`. + +--- + +## ExposedType Classes + +### ExposedType\ Class + +Abstract base class to declare exposed entities and fields for replication. + +#### Important Members: + +- `Expose(...)`: Define properties to expose. +- `AddDependency(...)`: Declare dependencies for associated entities. +- `AddChildDependency(...)`: Define reverse (one-to-many) entity relationships. +- `ExposeEverything()`: Automatically expose all suitable properties. +- `Filter()` and `FilterAsync()`: Customize which records get replicated. + +### Example: + +```csharp +public class UserExposedType : ExposedType +{ + public override void Define() + { + Expose(u => u.FirstName); + Expose(u => u.LastName); + Expose(u => u.Email); + AddDependency(u => u.Department); + } +} +``` + +--- + +### HardDeletableExposedType Class + +Derives from `ExposedType`. Overrides the soft-deletion behavior to implement hard deletes instead. + +- `IsSoftDeleteEnabled`: Returns `false`, indicating the records are physically deleted on replication targets. + +--- + +### NakedExposedType & HardDeletableNakedExposedType Classes + +Automatically expose all fields of the entity without explicitly defining each property. + +```csharp +public class NakedUserExposedType : NakedExposedType { } // Exposes all fields implicitly. +``` + +--- + +## ExposedField Classes + +### ExposedPropertyInfo and CustomExposedField + +These classes manage exposure of domain properties and derived or custom computed properties for replication. + +- `ExposedPropertyInfo`: Automatically manages serialization of regular domain properties. +- `CustomExposedField`: For custom, computed, or derived properties. + +### Example: + +```csharp +Expose("FullName", user => user.FirstName + " " + user.LastName); +``` + +--- + +## Replication Attributes + +### ExportDataAttribute + +Marks data types (exposed types) to be included in endpoint replication. + +```csharp +[ExportData(typeof(UserExposedType))] +public class UserEndpoint : SourceEndpoint { } +``` + +### EventBusAttribute + +Defines the replication queue endpoint URLs. + +```csharp +[EventBus(environment: "Production", url: "https://sqs.myqueue.aws.amazon.com")] +public class UserEndpoint : SourceEndpoint { } +``` + +--- + +## Source and Destination Endpoints + +### SourceEndpoint + +Publishes data changes from source domain models onto replication queues. + +#### Key Features: + +- `Publish()`: Starts publishing handlers. +- `UploadAll()`: Full upload (e.g., for replication initialization or refreshes). +- Automatically handles saving events and publishes messages. + +### DestinationEndpoint + +Consumes and applies replication data messages to local databases. + +#### Key Features: + +- `Subscribe()`: Starts listening to queue messages to replicate incoming changes. +- Event-driven architecture provides replication events (`Saving`, `Saved`, `Deleting`, `Deleted`) to hook custom logic. + +### EndpointSubscriber + +Internal class handling deserialization, importing, and updating local database records based on messages. + +--- + +## Replication Messages + +Structured communication objects encapsulating replicated data: + +- `ReplicateDataMessage`: Contains serialized data, metadata, and operation type (delete or update). +- `RefreshMessage`: Signals complete data refresh per entity type. + +--- + +## Examples + +### Defining a Simple Exposed Type + +```csharp +class ProductExposedType : ExposedType +{ + public override void Define() + { + Expose(p => p.Name); + Expose(p => p.Price); + } +} +``` + +### Setting Up a Source Endpoint to Publish Data + +```csharp +[ExportData(typeof(ProductExposedType))] +public class ProductsSourceEndpoint : SourceEndpoint +{ + public void StartReplication() + { + Publish(); + } +} +``` + +### Destination Endpoint Subscription and Handling + +```csharp +public class ProductsDestEndpoint : DestinationEndpoint +{ + public ProductsDestEndpoint(DbContext dbContext) + : base(typeof(Product).Assembly, dbContext) + { + Register(typeof(Product).FullName); + } + + public async Task StartListening() + { + Saved += async e => Console.WriteLine($"Saved Product {e.Instance.GetId()}"); + await Subscribe(); + } +} +``` + +--- + +## Configuration + +### EventBus URL configuration example (`appsettings.json`): + +```json +{ + "EventBus": { + "QueueUrls": { + "ProductsSourceEndpoint": "https://sqs.myqueue.aws.amazon.com/123456789/products" + } + } +} +``` + +--- + +## Notes + +- Ensure proper EF data annotations (`[Table]`, `[Column]`, `[DynamoDBHashKey]`) are used for accurate mapping. +- Carefully review soft/hard delete behavior as it impacts data integrity. +- Monitor replication events to manage consistency and handle exceptions effectively. \ No newline at end of file From 729d7df2f250440aa069783a9667ac195ed2574a Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Mon, 3 Mar 2025 14:19:00 +0000 Subject: [PATCH 31/68] Olive.Entities.Data.MySql doc --- docs/Entities/Olive.Entities.Data.MySql.md | 207 +++++++++++++++++++++ 1 file changed, 207 insertions(+) create mode 100644 docs/Entities/Olive.Entities.Data.MySql.md diff --git a/docs/Entities/Olive.Entities.Data.MySql.md b/docs/Entities/Olive.Entities.Data.MySql.md new file mode 100644 index 000000000..e85710e9a --- /dev/null +++ b/docs/Entities/Olive.Entities.Data.MySql.md @@ -0,0 +1,207 @@ +# Olive.Entities.Data.MySql + +## Overview +The Olive Framework's MySQL Adaopter provides seamless integration of MySQL databases with Olive's data access infrastructure. It offers implementations for connection management, SQL generation, and database commands optimized specifically for MySQL. + +--- + +## Table of Contents +- [Key Components](#key-components) +- [MySqlManager Class](#mysqlmanager-class) +- [MySqlCommandGenerator Class](#mysqlcommandgenerator-class) +- [MySqlConnectionStringProvider Class](#mysqlconnectionstringprovider-class) +- [MySqlDataAccessExtensions](#mysqldataaccessextensions-class) +- [MySqlCriterionGenerator Class](#mysqlcriteriongenerator-class) +- [ParameterFactory](#parameterfactory-class) +- [Examples](#examples) +- [Configuration](#configuration) + +--- + +## Key Components +- **MySqlManager**: Manages database operations including executing scripts, clearing pools, and checking database existence. +- **MySqlCommandGenerator**: Generates MySQL-specific SQL commands for commonly used queries. +- **MySqlConnectionStringProvider**: Provides specialized MySQL connection strings. +- **MySqlDataAccessExtensions**: Extension methods for simplified MySQL integration with Olive DataAccess. +- **MySqlCriterionGenerator**: Generates criteria expressions tailored to MySQL. +- **ParameterFactory**: Generates MySQL-compatible parameters for database commands. + +--- + +## MySqlManager Class + +Manages common database operations for MySQL, such as executing commands, checking if databases exist, and clearing connection pools. + +#### Methods: + +- **ClearConnectionPool()** + Clears all MySQL connection pools. + +- **Delete(string database)** + Drops the specified database. + +- **Execute(string script, string database = null)** + Executes provided SQL script. + +- **Exists(string database, string filePath)** + Checks if the specified database exists. + +**Example Usage:** +```csharp +var manager = new MySqlManager(); + +// Execute script +manager.Execute("CREATE TABLE Users (ID INT PRIMARY KEY);", "MyDatabase"); + +// Check database Existence +bool exists = manager.Exists("MyDatabase", "/var/lib/mysql/MyDatabase"); +``` + +--- + +## MySqlCommandGenerator Class + +Generates SQL commands tailored specifically for MySQL databases. + +#### Methods: + +- **GenerateSelectCommand(...)** + Generates SELECT statements including pagination and sorting. + +- **GeneratePagination(...)** + Adds pagination logic (`LIMIT`) to queries. + +- **SafeId(string id)** + Safely escapes identifiers with backticks. + +**Example Usage:** +```csharp +var generator = new MySqlCommandGenerator(); +string sql = generator.GenerateSelectCommand(query, "Users", "*"); +``` + +--- + +## MySqlConnectionStringProvider Class + +Customizes connection strings specifically for MySQL databases, including additional MySQL-specific settings. + +#### Methods: + +- **GetConnectionString(string key = "Default")** + Returns a connection string optimized for MySQL. + +**Example Usage:** +```csharp +var provider = new MySqlConnectionStringProvider(); +string connString = provider.GetConnectionString(); +``` + +--- + +## MySqlDataAccessExtensions Class + +Extension methods allowing simplified MySQL integration with Olive's DataAccessOptions. + +#### Methods: + +- **MySql()** + Adds MySQL-specific implementations of command generators and parameter factories. + +**Example Usage:** +```csharp +dataAccessOptions.MySql(); +``` + +--- + +## MySqlCriterionGenerator Class + +Generates SQL snippets for database criteria expressions that match MySQL syntax. + +**Example Usage (internal use):** +This class is used internally by Olive's data query mechanism. + +--- + +## ParameterFactory Class + +Generates parameters compatible with MySQL commands. Used internally by Olive's data access layer. + +--- + +## Examples + +### Configuring DataAccess to Use MySQL + +```csharp +DataAccess.Configure(options => options.MySql()); +``` + +### Executing SQL Script Using MySqlManager + +```csharp +// Initialize MySqlManager +var manager = new MySqlManager(); + +// Execute commands on database +manager.Execute("DROP TABLE IF EXISTS Users; CREATE TABLE Users(ID INT PRIMARY KEY AUTO_INCREMENT);", "MyDatabase"); +``` + +### Generating Select Query Using MySqlCommandGenerator + +```csharp +var generator = new MySqlCommandGenerator(); + +var query = new DatabaseQuery(typeof(User)) +{ + PageStartIndex = 0, + TakeTop = 10 +}; + +string sqlCommand = generator.GenerateSelectCommand(query, "Users", "ID, UserName"); +``` + +--- + +## Configuration + +To configure your Olive application for MySQL, ensure your app configuration (`appsettings.json`, `web.config`, etc.) includes a valid MySQL connection string: + +```json +{ + "ConnectionStrings": { + "Default": "Server=myServerAddress;Database=myDataBase;Uid=myUsername;Pwd=myPassword;" + } +} +``` + +Furthermore, enable MySQL extensions as follows: + +```csharp +DataAccess.Configure(o => o.MySql()); +``` + +--- + +## Important Notes + +- Ensure the MySql.Data library is available in your project. +- Use backticks (`) to safely escape column and table names, as provided by the `SafeId` method in `MySqlCommandGenerator`. +- The provided classes handle internal tasks of Olive Framework and typically do not require manual invocation in everyday use. + +--- + +## Dependencies + +Make sure you're referencing MySql.Data library to use Olive Entities MySQL Adapter: + +- **NuGet Package Installation:** +```bash +Install-Package MySql.Data +``` + +--- + +## Exception Handling +The methods are wrapped to throw meaningful exceptions in case of failures (e.g., cannot establish database connections or invalid SQL scripts). Use try-catch blocks in your application as needed. \ No newline at end of file From b412e748d5602fa588bfd76d474f67a4ec1c2ac7 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Mon, 3 Mar 2025 14:45:35 +0000 Subject: [PATCH 32/68] Olive.Entities.Data.PostgreSQL doc --- .../Olive.Entities.Data.PostgreSQL.md | 131 ++++++++++++++++++ 1 file changed, 131 insertions(+) create mode 100644 docs/Entities/Olive.Entities.Data.PostgreSQL.md diff --git a/docs/Entities/Olive.Entities.Data.PostgreSQL.md b/docs/Entities/Olive.Entities.Data.PostgreSQL.md new file mode 100644 index 000000000..e626e864f --- /dev/null +++ b/docs/Entities/Olive.Entities.Data.PostgreSQL.md @@ -0,0 +1,131 @@ +# Olive.Entities.Data.PostgreSQL + +## Overview +Olive's PostgreSQL Data Adapter provides seamless integration and compatibility of PostgreSQL databases within the Olive Entities framework. It includes specialized implementations for generating PostgreSQL-specific SQL commands and criteria handling, enabling efficient data access through the Olive data access layer. + +--- + +## Table of Contents +- [Key Components](#key-components) +- [PostgreSqlCommandGenerator Class](#postgresqlcommandgenerator-class) +- [PostgreSqlCriterionGenerator Class](#postgresqlcriteriongenerator-class) +- [PostgreSqlDataAccessExtensions Class](#postgresqldataaccessextensions-class) +- [Usage Examples](#usage-examples) +- [Configuration](#configuration) +- [Dependencies](#dependencies) + +--- + +## Key Components +- **PostgreSqlCommandGenerator**: Generates PostgreSQL-compatible SQL commands. +- **PostgreSqlCriterionGenerator**: Handles query conditions specifically tailored for PostgreSQL. +- **PostgreSqlDataAccessExtensions**: Provides simplified integration within Olive's DataAccess configuration. + +--- + +## PostgreSqlCommandGenerator Class + +Generates PostgreSQL-specific SQL command strings for querying entities, providing efficient queries with correct pagination and syntax. + +### Methods: + +- **GenerateSelectCommand(IDatabaseQuery iquery, string tables, string fields)** + Creates a SELECT statement including fields, tables, criteria, sorting, and pagination. + +- **GeneratePagination(IDatabaseQuery query)** + Appends LIMIT and OFFSET clauses specific to PostgreSQL. + +- **SafeId(string id)** + Properly escapes identifiers using double-quotes (`"`), following PostgreSQL conventions. + +- **UnescapeId(string id)** + Removes escaping quotes from identifiers. + +### Example: +```csharp +var generator = new PostgreSqlCommandGenerator(); + +var query = new DatabaseQuery(typeof(User)) +{ + PageStartIndex = 20, + TakeTop = 10 +}; + +string selectQuery = generator.GenerateSelectCommand(query, "Users", "ID, Name, Email"); +``` + +--- + +## PostgreSqlCriterionGenerator Class + +Generates PostgreSQL-specific criteria SQL fragments for querying databases. + +- Internally used by the Olive Framework. +- Uses double-quotes (`"`) for identifier quoting. + +Implementation example (for reference, normally internal): + +```csharp +protected override string ToSafeId(string id) => $"\"{id}\""; +protected override string UnescapeId(string id) => id.Trim('\"'); +``` + +--- + +## PostgreSqlDataAccessExtensions Class + +Enables easy integration of PostgreSQL provider and command generation in DataAccess configuration via extension methods. + +### Methods: +- **PostgreSql()**: Configures Olive's data access layer to use PostgreSQL. +- +--- + +## Usage Examples + +### Generating a PostgreSQL SELECT Command +```csharp +var commandGenerator = new PostgreSqlCommandGenerator(); + +var query = new DatabaseQuery(typeof(Product)) + { + TakeTop = 5, + PageStartIndex = 10 + }; + +string sqlCommand = commandGenerator.GenerateSelectCommand(query, "Products", "*"); +// Resulting SQL: +// SELECT * FROM Products LIMIT 5 OFFSET 10 +``` + +--- + +## Configuration + +Setup a PostgreSQL connection string in your application's settings (e.g. `appsettings.json`): + +```json +{ + "ConnectionStrings": { + "Default": "Host=myserver;Database=mydb;Username=mylogin;Password=mypass" + } +} +``` + +--- + +## Dependencies + +This adapter requires adding the Npgsql library, an open-source .NET data provider for PostgreSQL: + +```bash +Install-Package Npgsql +``` + +--- + +## Important Notes + +- PostgreSQL identifiers use double-quotes (`"`) to preserve case sensitivity. +- Ensure proper escaping to avoid syntax errors or injection vulnerabilities. +- Olive automatically handles correct escaping through methods provided in command and criteria generators. \ No newline at end of file From 443173767f1d8644114c391dffe82ee7a3f903b1 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Mon, 3 Mar 2025 16:00:24 +0000 Subject: [PATCH 33/68] Olive.Entities.Data.Replication.DataGenerator.UI doc --- ...ities.Data.Replication.DataGenerator.UI.md | 155 ++++++++++++++++++ 1 file changed, 155 insertions(+) create mode 100644 docs/Entities/Olive.Entities.Data.Replication.DataGenerator.UI.md diff --git a/docs/Entities/Olive.Entities.Data.Replication.DataGenerator.UI.md b/docs/Entities/Olive.Entities.Data.Replication.DataGenerator.UI.md new file mode 100644 index 000000000..0247f249c --- /dev/null +++ b/docs/Entities/Olive.Entities.Data.Replication.DataGenerator.UI.md @@ -0,0 +1,155 @@ +# Olive.Entities.Data.Replication.DataGenerator.UI + +## Overview + +The Olive Replication Data Generator UI provides developer commands and UI elements for easily generating mock data for entities managed by Olive's replication module. It facilitates quick creation, viewing, and management of test data, significantly streamlining the development and testing processes. + +--- + +## Table of Contents: + +- [Key Components](#key-components) +- [ReplicationDataGeneratorUICommand](#replicationdatageneratoruicommand) +- [Serialization Resolver](#replicationmessageserializationresolver) +- [IModuleGenerator Interface](#imodulegenerator-interface) +- [ModuleGenerators](#module-generators) + - [BaseModuleGenerator](#basemodulegenerator-class) + - [FormModuleGenerator](#formmodulegenerator-class) + - [FormSubmitGenerator](#formsubmitgenerator-class) + - [ListModuleGenerator](#listmodulegenerator-class) +- [Extension Methods](#extension-methods) +- [Usage Examples](#usage-examples) + +--- + +## Key Components + +- **ReplicationDataGeneratorUICommand**: Developer command to invoke or render different UI modules. +- **IModuleGenerator Interface**: Defines methods for UI module generation (list, form, submit). +- **Serialization resolver**: Ensures entity serialization for replication messages. +- **BaseModuleGenerator & subclasses**: Generate HTML view components dynamically based on entity definitions. +- **Extensions methods**: Easily integrates mocking capabilities into services and dependency injection. + +--- + +## ReplicationDataGeneratorUICommand + +Implements `IDevCommand`, allowing developers to generate HTML UI based on Olive replication entities directly through configured development commands. + +### Methods and Properties: + +- `Name`: Command identifier as `"generate-{Namespace}-{Type}"`. +- `Title`: Human-readable identifier. +- `IsEnabled()`: Checks if endpoint is properly initialized. +- `Run()`: Generates the appropriate UI based on user HTTP requests (list views, form views, or form submissions). + +### Example: + +```csharp +var command = new ReplicationDataGeneratorUICommand("MyNamespace", "User"); +command.SetEndpoint(myEndpoint); +string content = await command.Run(); +``` + +--- + +## ReplicationMessageSerializationResolver + +Custom contract resolver derived from `DefaultContractResolver` used to serialize entities into replication messages: + +- Ensures only writable properties are serialized. +- Primarily used internally when generating replication messages for mock data. + +--- + +## IModuleGenerator Interface + +Defines a consistent way to generate UI modules: + +- `Render(string nameSpace, string typeName)`: Generate HTML content for the entity interaction (list, edit form, submit). + +Implemented by: +- `FormModuleGenerator` +- `FormSubmitGenerator` +- `ListModuleGenerator` + +--- + +## Module Generators + +Common classes extending `BaseModuleGenerator` to generate different UI modules dynamically. + +### BaseModuleGenerator Class + +- Shared functionalities like rendering HTML basics, fetching entity type information, and extracting property definitions. +- Provides base rendering logic used across sub-generators. + +### FormModuleGenerator Class + +Generates an HTML form for creating new entity instances: + +- Dynamically creates fields based on entity type properties. +- Supports different HTML input types based on entity property types (text, checkbox, etc.). + +### FormSubmitGenerator Class + +Processes submitted form data, saves the entity to the database through the replication system: + +- Handles form submission logic, data validation, and replication message generation. +- Provides error handling and validation feedback mechanisms. + +### ListModuleGenerator Class + +Displays existing entity records as an HTML table: + +- Dynamically constructs tables based on entity property definitions. +- Provides an intuitive view to examine existing mock data. + +--- + +## Extension Methods + +### Usage within Startup.cs: + +- **AddMockEndpoint\(...)**: Registers defined entities into the application’s service collection for mock UI generation. + +- **UseMockEndpoint(...)**: Associates the configured endpoint with registered developer commands within the application’s runtime provider. + +Example: +```csharp +// Register in ConfigureServices method +services.AddMockEndpoint("MyNamespace"); + +// Configure the endpoint during application startup +provider.UseMockEndpoint(myDestinationEndpoint, "MyNamespace"); +``` + +--- + +## Usage Examples + +### Using the Generated Developer Commands: + +Via browser or tool: +- To see a list view: + ``` + /cmd/generate-MyNamespace-User + ``` +- To access a form view directly for creating new data: + ``` + /cmd/generate-MyNamespace-User?action=new + ``` + +--- + +## Important Notes: + +- Proper endpoint initialization (`SetEndpoint()` or using dependency injection) is required. +- HTML rendering follows Bootstrap for styling and responsiveness. +- Extension methods simplify registrations and wiring; always use them for easier maintainability. + +--- + +## Exception Handling: + +Appropriate validation messages and exception catching are provided in form submission to aid developer debugging during UI data generation sessions. \ No newline at end of file From 966c9d7754be852a0addaf684401199c49c7ad72 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Mon, 3 Mar 2025 16:31:04 +0000 Subject: [PATCH 34/68] Olive.Entities.Data.Replication.Mvc.Extensions doc --- ...ntities.Data.Replication.Mvc.Extensions.md | 150 ++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 docs/Entities/Olive.Entities.Data.Replication.Mvc.Extensions.md diff --git a/docs/Entities/Olive.Entities.Data.Replication.Mvc.Extensions.md b/docs/Entities/Olive.Entities.Data.Replication.Mvc.Extensions.md new file mode 100644 index 000000000..f91623298 --- /dev/null +++ b/docs/Entities/Olive.Entities.Data.Replication.Mvc.Extensions.md @@ -0,0 +1,150 @@ +# Olive.Entities.Data.Replication.Mvc.Extensions + +## Overview + +The Olive Entities Data Replication provides extension methods to seamlessly integrate Olive's data replication system into ASP.NET Core applications. It simplifies registering and configuring replication sources (publishers) and destinations (subscribers) within your application's middleware pipeline. + +--- + +## Table of Contents +- [Key Components](#key-components) +- [Extensions Methods](#extension-methods) + - [RegisterSubscriber](#registersubscriber) + - [RegisterPublisher](#registerpublisher) +- [Examples](#examples) +- [Configuration Options](#configuration-options) + +--- + +## Key Components + +The provided extension methods handle common tasks: + +- **RegisterSubscriber**: Sets up a replication subscriber endpoint, enabling data synchronization from external sources. +- **RegisterPublisher**: Configures a replication publisher endpoint that can expose data to other subscribed applications. + +--- + +## Extension Methods + +### RegisterSubscriber + +Registers a replication subscriber (destination endpoint), setting up automatic data pulling from configured replication queues. + +**Syntax:** + +```csharp +public static IApplicationBuilder RegisterSubscriber(this IApplicationBuilder app, Assembly domainAssembly) + where T : DestinationEndpoint +``` + +**Features:** + +- In 'MultiServer' mode, registers a `BackgroundJob` to periodically pull updates every minute (`* * * * *`). +- In 'SingleServer' mode, directly subscribes to replication events on application startup. +- Sets up an HTTP mapping (`/olive-endpoints/{endpoint}`) allowing manual triggering of replication. + +**Usage:** + +```csharp +app.RegisterSubscriber(typeof(MyEntity).Assembly); +``` + +--- + +### RegisterPublisher + +Registers a replication publisher (source endpoint), enabling entities to be pushed and refreshed from publisher applications. + +**Syntax:** + +```csharp +public static IApplicationBuilder RegisterPublisher(this IApplicationBuilder app, SourceEndpoint endpoint); +public static IApplicationBuilder RegisterPublisher(this IApplicationBuilder app) where TSourceEndpoint : SourceEndpoint, new(); +``` + +**Features:** + +- Publishes changes for registered entities on the replication event bus. +- Optionally registers dump URLs for manual data uploading and refreshing. +- Provides a summary URL (`/olive/entities/replication/dump/all`) to list all registered endpoints for debugging or manual triggers. + +**Usage:** + +```csharp +app.RegisterPublisher(); +``` + +or with a created instance: + +```csharp +var myEndpoint = new MySourceEndpoint(); +app.RegisterPublisher(myEndpoint); +``` + +--- + +## Examples + +### 1. Configuring a Subscriber Endpoint + +In Startup.cs, `Configure` method: + +```csharp +app.RegisterSubscriber(typeof(MyEntity).Assembly); +``` + +### 2. Configuring a Publisher Endpoint + +In Startup.cs, `Configure` method: + +```csharp + app.RegisterPublisher(); +``` + +Creating custom publisher instance with specific configurations: + +```csharp +var publisher = new CustomSourceEndpoint(); +app.RegisterPublisher(publisher); +``` + +--- + +## Configuration Options + +### AppSettings Configuration (`appsettings.json`): + +```json +"DataReplication": { + "Mode": "MultiServer", + "AllowDumpUrl": true, + "DumpUrl": "/olive/entities/replication/dump/" +} +``` + +| Option | Description | Default | +|--------|---------------------------------------------------------------------------------------------|-------------------------------------| +| Mode | Defines mode: Set "MultiServer" for distributed scenarios or "SingleServer" otherwise. | **Required** (no default value) | +| AllowDumpUrl | Enables the Dump URLs to refresh or upload all data via browser or manual HTTP request. | true | +| DumpUrl | Base URL prefix for exposing refresh endpoints. | /olive/entities/replication/dump/ | + +--- + +## Important Notes + +- Ensure correct configuration of replication modes (`MultiServer` or `SingleServer`) before deployment. +- Dump URL refresh actions should typically remain enabled only in development or secured environments due to their sensitive nature. +- Logs are generated for each action and request, providing detailed audit trails and debugging information. + +--- + +## Exception Handling + +Configuration missteps, such as omitting `"DataReplication:Mode"`, will raise exceptions at runtime startup. Clearly defined exception messages guide straightforward configuration troubleshooting. + +Example exception messages: + +```text +Exception: DataReplication:Mode has not been specified. Options: MultiServer, SingleServer +``` \ No newline at end of file From 0d279be52fb66af1b5223eb311d0f2818e8088b2 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Mon, 3 Mar 2025 16:54:14 +0000 Subject: [PATCH 35/68] Olive.Entities.Data.Replication.QueueUrlProvider doc --- ...ities.Data.Replication.QueueUrlProvider.md | 115 ++++++++++++++++++ 1 file changed, 115 insertions(+) create mode 100644 docs/Entities/Olive.Entities.Data.Replication.QueueUrlProvider.md diff --git a/docs/Entities/Olive.Entities.Data.Replication.QueueUrlProvider.md b/docs/Entities/Olive.Entities.Data.Replication.QueueUrlProvider.md new file mode 100644 index 000000000..d05adadb2 --- /dev/null +++ b/docs/Entities/Olive.Entities.Data.Replication.QueueUrlProvider.md @@ -0,0 +1,115 @@ +# Olive.Entities.Data.Replication.QueueUrlProvider + +## Overview + +The Olive Framework provides a specialized utility for retrieving replication queue URLs dynamically through configuration. The `UrlProvider` class centralizes URL resolution logic for replication endpoints, facilitating cleaner and more maintainable configuration across various environments and deployments. + +--- + +## Table of Contents + +- [Class Overview](#class-overview) +- [Methods](#methods) +- [Usage Examples](#usage-examples) +- [Configuration](#configuration) +- [Notes](#notes) + +--- + +## Class Overview + +### `UrlProvider` Class + +The `UrlProvider` class simplifies retrieval of queue URLs for Olive's data replication endpoints. It reads configuration settings dynamically based on given endpoint types and provides default behavior to assist during development. + +--- + +## Methods + +### `GetUrl` + +Retrieves the URL configured for a specified replication endpoint type from configuration. It attempts multiple configuration key patterns and falls back to a clearly marked development-only URL if none are configured. + +#### Signature: + +```csharp +public static string GetUrl(Type type); +``` + +#### Behavioral Details: + +- **First attempt:** Tries to fetch from configuration with key pattern: + ``` + DataReplication:{Full_Type_Name_With_Underscores}:Url + ``` +- **Second attempt (fallback):** Tries to fetch from configuration with key pattern: + ``` + DataReplication:{Full.Type.Name}:Url + ``` +- **Default/Development fallback** (if both attempts fail): + ``` + FOR_DEVELOPMENT_ONLY_DataReplication_{Full.Type.Name} + ``` + +#### Example keys it attempts: + +For type `My.Namespace.UserEndpoint`: +``` +DataReplication:My_Namespace_UserEndpoint:Url +DataReplication:My.Namespace.UserEndpoint:Url +``` + +--- + +## Usage Examples + +### Obtaining a URL dynamically for replication: + +```csharp +Type endpointType = typeof(MyNamespace.MyUserEndpoint); +string replicationUrl = UrlProvider.GetUrl(endpointType); +``` + +_If configured correctly, returns something like:_ + +``` +https://sqs.region.amazonaws.com/1234567890/my-user-endpoint +``` + +If nothing is configured (_development scenario_), it returns: + +``` +FOR_DEVELOPMENT_ONLY_DataReplication_MyNamespace.MyUserEndpoint +``` + +--- + +## Configuration + +In your application's `appsettings.json` or other configuration sources, you should set the Queue URL to connect your replication endpoints: + +### Example configuration: + +```json +{ + "DataReplication": { + "My_Namespace_UserEndpoint": { + "Url": "https://sqs.region.amazonaws.com/1234567890/my-user-endpoint" + }, + "My.Namespace.UserEndpoint": { + "Url": "https://sqs.region.amazonaws.com/1234567890/my-user-endpoint-alt" + } + } +} +``` + +**Note**: +Either configuration pattern (`_` or `.`) can be used. The provider checks both, starting with the underscore replacement convention first. + +--- + +## Notes + +- The "FOR_DEVELOPMENT_ONLY" default prefix highlights clearly when URLs are not configured, assisting accurate identification and avoidance of unintended production deployments. +- It is recommended to configure real environment-specific URLs for staging and production environments explicitly in your configuration files. +- UrlProvider centralizes logic for URL retrieval, ensuring consistency across your replication management solutions. \ No newline at end of file From 0aec8d9b195a734b80f8997e3e4bbb611e7fb598 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 4 Mar 2025 07:22:32 +0000 Subject: [PATCH 36/68] Olive.Entities.Data.Replication doc --- docs/Api/Olive.Entities.Data.Replication.md | 293 ++++++++++++++++++ .../Olive.Entities.Data.Replication.md | 220 +++++++++++++ 2 files changed, 513 insertions(+) create mode 100644 docs/Api/Olive.Entities.Data.Replication.md create mode 100644 docs/Entities/Olive.Entities.Data.Replication.md diff --git a/docs/Api/Olive.Entities.Data.Replication.md b/docs/Api/Olive.Entities.Data.Replication.md new file mode 100644 index 000000000..f8fe5e9f3 --- /dev/null +++ b/docs/Api/Olive.Entities.Data.Replication.md @@ -0,0 +1,293 @@ +# Olive.Entities.Data.Replication + +## Overview + +Olive Data Replication provides an efficient, reliable, and robust mechanism for synchronizing data between different microservices. By leveraging event-driven architecture and hosted replication queues, Olive ensures that data dependency between services is handled gracefully, improving system resilience and performance. + +--- + +## Table of Contents + +- [Introduction](#introduction) +- [Data Replication vs. API-Based Integration](#data-replication-vs-api-based-integration) +- [Essential Components](#essential-components) + - [ExposedType Classes](#exposedtype-classes) + - [Endpoint Classes](#endpoint-classes) + - [SourceEndpoint](#sourceendpoint) + - [DestinationEndpoint](#destinationendpoint) + - [Attributes](#attributes) + - [ExportDataAttribute](#exportdataattribute) + - [EventBusAttribute](#eventbusattribute) + - [Replication Messages](#replication-messages) +- [Implementation Example](#implementation-example) +- [Starting the Endpoint Engine](#starting-the-endpoint-engine) +- [Generating Endpoint Proxy Packages](#generating-endpoint-proxy-packages) +- [Advanced Configurations and Tips](#advanced-configurations-and-tips) +- [Exception Handling and Logging](#exception-handling-and-logging) +- [Configuration Options](#configuration-options) +- [Dependencies](#dependencies) + +--- + +## Introduction + +In microservices architectures, services often depend on data maintained by other services. Traditionally, APIs were used to share this data, but this approach has limitations around performance, resiliency, and data freshness. Olive Data Replication solves these challenges by replicating relevant data asynchronously between microservices using event-driven queues. + +--- + +## Data Replication vs. API-Based Integration + +| Aspect | API Approach | Olive Data Replication Approach | +|------------------------|-----------------------------------------------------------------------|--------------------------------------------------| +| **Performance** | Frequent calls can become slow | Local data access; much faster performance | +| **Resiliency** | Consumer outages when publisher is unavailable | Replicated data available independent of publisher availability | +| **Data Freshness** | Caching introduces data lag | Data changes are immediately pushed to consumers | +| **Complexity** | Cache invalidation and synchronization complexity | Simplified, event-driven approach reduces complexity | + +--- + +## Essential Components + +### ExposedType Classes + +Define and expose entity fields for replication: + +- `ExposedType`: explicitly exposes chosen fields. +- `NakedExposedType`: implicitly exposes all fields. +- `HardDeletableExposedType`: allows physical deletion of records on replication targets. + +**Example:** + +```csharp +public class Customer : ExposedType +{ + public override void Define() + { + Expose(x => x.Name); + Expose(x => x.Email); + } +} +``` + +### Endpoint Classes + +Endpoints manage replication publishing and subscribing. + +#### SourceEndpoint + +Publishes entity changes to replication queues, allowing replication subscribers to update their data. Each update triggers an asynchronous message to an event bus queue. + +#### DestinationEndpoint + +Receives and processes data replication messages, updating the consumer's local data copy. + +--- + +### Attributes + +#### ExportDataAttribute + +Indicates specific exposed data types to include in a source endpoint for replication. + +```csharp +[ExportData(typeof(Customer))] +public class CustomersSourceEndpoint : SourceEndpoint { } +``` + +#### EventBusAttribute + +Defines the event bus queue endpoints for publishing or subscribing. + +```csharp +[EventBus("Production", "https://sqs.aws-region.amazonaws.com/account-id/queue.fifo")] +public class CustomersSourceEndpoint : SourceEndpoint { } +``` + +--- + +### Replication Messages + +- `ReplicateDataMessage`: Contains serialized entity data. +- `RefreshMessage`: Used to initiate a refresh of replicated data from publishers to consumers. + +Example message structure: + +```json +{ + "TypeFullName": "MyNamespace.Customer", + "Entity": "{ \"ID\": \"123\", \"Name\": \"Alice\" }", + "CreationUtc": "2023-01-01T00:00:00Z", + "ToDelete": false, + "IsClearSignal": false +} +``` + +--- + +## Implementation Example + +Consider a "Customer Service" (publisher) exposing customer data and an "Order Service" (consumer) needing customer information to associate with orders. + +### Publisher-side Implementation: + +```csharp +namespace CustomerService +{ + [EventBus(Environments.Production, "https://sqs.aws-region.amazonaws.com/account/CustomerService-OrdersEndpoint.fifo")] + public class OrdersEndpoint : SourceEndpoint + { + class Customer : ExposedType + { + protected override void Define() + { + Expose(x => x.Email); + Expose(x => x.Name); + } + } + } +} +``` + +--- + +## Starting the Endpoint Engine + +In the `Startup.cs` of the publisher (Customer Service), invoke Publish: + +```csharp +public override async Task OnStartUpAsync(IApplicationBuilder app) +{ + await new CustomerService.OrdersEndpoint().Publish(); +} +``` + +In the consumer (Order Service), initiate subscription: + +```csharp +public override async Task OnStartUpAsync(IApplicationBuilder app) +{ + await new CustomerService.OrdersEndpoint(typeof(CustomerService.Order).Assembly).Subscribe(); +} +``` + +--- + +## Generating Endpoint Proxy Packages + +Using the `generate-data-endpoint-proxy` tool (a global dotnet tool), you can generate nuget packages for endpoint consumption. + +### Tool Installation: + +```batch +dotnet tool install -g generate-data-endpoint-proxy +``` + +### Generating Packages: + +```batch +generate-data-endpoint-proxy /assembly:"path\website.dll" /dataEndpoint:Namespace.OrdersEndpoint /out:"c:\generated-packages" +``` + +Generated packages: + +- **CustomerService.OrdersEndpoint**: Reference from consumer service for subscription. +- **CustomerService.OrdersEndpoint.MSharp**: Model definitions for consumer’s data layer. + +--- + +## Advanced Configurations and Tips + +### Expose All Fields Automatically (Shortcut): + +```csharp +public class Customer : NakedExposedType { } +``` + +### Custom Fields with Mappings: + +```csharp +protected override void Define() +{ + Expose(x => x.Name); + Expose("Address", x => $"{x.AddressLine1}, {x.City}, {x.PostalCode}"); +} +``` + +### Filtering Exposed Records: + +Only expose certain records via overriding filter method: + +```csharp +protected override bool Filter(Domain.Customer customer) => customer.Status == CustomerStatus.Approved; +``` + +For **async filtering**, override `FilterAsync`. + +--- + +## Exception Handling and Logging + +- Detailed exceptions for serialization and transmission issues. +- Comprehensive logging available through Olive's built-in logging mechanisms. + +Example logged exception message: + +``` +Exception: Could not serialize field Email on type CustomerService.Domain.Customer +``` + +--- + +## Configuration Options + +Provide necessary configurations via appsettings.json: + +```json +{ + "DataReplication": { + "Mode": "MultiServer", + "AllowDumpUrl": true, + "DumpUrl": "/olive/entities/replication/dump/", + "CustomerService_OrdersEndpoint": { + "Url": "https://sqs.aws-region.amazonaws.com/account-id/CustomerService-OrdersEndpoint.fifo" + } + } +} +``` + +| Key | Description | Default | +|----------------|-----------------------------------------------------|--------------------------| +| **Mode** | Specifies operational mode: SingleServer/MultiServer| Required | +| **AllowDumpUrl**| Enables manual refresh URLs (dev only) | true | +| **DumpUrl** | URL prefix for manual refresh endpoints | "/olive/…/replication/dump/" | + +--- + +## Dependencies + +- **NuGet Packages**: + +```bash +Install-Package Olive.Entities +Install-Package Newtonsoft.Json +``` + +- **Tools**: + +```batch +dotnet tool install -g generate-data-endpoint-proxy +``` + +--- + +## Data Replication Overall Architecture + +![Olive Data Replication Overview](DataReplicationOverview.png) + +--- + +## Important notes: + +- Ensure entities can safely handle delete scenarios. Soft deletion is highly recommended. +- Hard deletion requires careful handling of associated entities at consumer endpoints. +- Actions exposed via manual URLs (`/dump/...`) should be restricted to development or secure environments. \ No newline at end of file diff --git a/docs/Entities/Olive.Entities.Data.Replication.md b/docs/Entities/Olive.Entities.Data.Replication.md new file mode 100644 index 000000000..f20851f1c --- /dev/null +++ b/docs/Entities/Olive.Entities.Data.Replication.md @@ -0,0 +1,220 @@ +# Olive.Entities.Data.Replication + +## Overview +Olive Entities Replication provides a structured and efficient mechanism for replicating and synchronizing entity data across multiple database instances or services. It leverages an event-driven architecture to enable seamless data replication, synchronization, and consistent maintenance of entities across different endpoints. + +--- + +## Table of Contents +- [Key Components](#key-components) +- [ExposedType Classes](#exposedtype-classes) + - [ExposedType\](#exposedtype-tdomain) + - [NakedExposedType](#nakedexposedtype) + - [HardDeletableExposedType](#harddeletableexposedtype) +- [Attributes](#attributes) + - [ExportDataAttribute](#exportdataattribute) + - [EventBusAttribute](#eventbusattribute) +- [Endpoint Classes](#endpoint-classes) + - [SourceEndpoint](#sourceendpoint) + - [DestinationEndpoint](#destinationendpoint) + - [EndpointSubscriber](#endpointsubscriber) +- [Replication Messages](#replication-messages) + - [ReplicateDataMessage](#replicatedatamessage) + - [RefreshMessage](#refreshmessage) +- [Examples](#examples) +- [Configuration](#configuration) + +--- + +## Key Components +- **Exposed Types** – Define entities and relationships targeted for replication. +- **Source and Destination Endpoints** – Handle publication and subscription of replication data. +- **Replication Messages** – Structure the replication data payloads. +- **Attributes** – Declaratively configure replication aspects of entities and endpoints. + +--- + +## ExposedType Classes + +### ExposedType\ + +Allows exposure of domain entity data to replication mechanism. + +#### Main Methods: + +- `Define()` – Declares exposed fields and relationships. +- `Expose(...)` – Selectively exposes entity properties. +- `ExposeEverything()` – Quickly exposes all fields for simple scenarios. +- `AddDependency(...)` – Links related entities to synchronize changes. + +**Example Usage:** +```csharp +class UserExposedType : ExposedType +{ + public override void Define() + { + Expose(u => u.Name); + Expose(u => u.Email); + AddDependency(u => u.Department); + } +} +``` + +### NakedExposedType + +A special version that auto-exposes all fields automatically. + +```csharp +class NakedUserType : NakedExposedType { } +``` + +### HardDeletableExposedType + +Overrides default soft-deletion behavior; used when entities must be physically deleted on target endpoints. + +--- + +## Attributes + +### ExportDataAttribute + +Specifies the export of certain exposed types at an endpoint level. + +```csharp +[ExportData(typeof(UserExposedType))] +public class UsersSourceEndpoint : SourceEndpoint { } +``` + +### EventBusAttribute + +Specifies the queue URL where replicated messages are published/subscribed, for different environment configurations. + +```csharp +[EventBus(environment: "Production", url: "https://sqs.aws.amazon.com/queue-url")] +public class UsersSourceEndpoint : SourceEndpoint { } +``` + +--- + +## Endpoint Classes + +### SourceEndpoint + +Publishes entity changes/events to replication queues, allowing downstream subscribers to react accordingly. + +#### Key Methods: +- `Publish()` – Starts publishing configured entities/events. +- `UploadAll()` – Manually triggers a full replication upload. + +### DestinationEndpoint + +Consumes replication data from queues and applies changes received from the source to the local database. + +#### Key Methods: +- `Subscribe()` – Begins listening for replication messages. +- `PullAll()` – Pulls all pending replication data from configured queues. + +### EndpointSubscriber + +Internal class used by DestinationEndpoint to deserialize and apply incoming replication messages. + +#### Main Functionalities: +- Implement deserialization logic. +- Manage change detection to minimize unnecessary database writes. +- Handle refresh requests and table clears during replication initialization. + +--- + +## Replication Messages + +### ReplicateDataMessage + +Core payload object, encapsulating entity data changes: +- Includes entity type, serialized data, timestamps, and delete indicators. + +```json +{ + "TypeFullName": "MyNamespace.User", + "Entity": "{ \"ID\": \"123\", \"Name\": \"John Doe\" }", + "CreationUtc": "2023-01-01T00:00:00Z", + "ToDelete": false, + "IsClearSignal": false +} +``` + +### RefreshMessage + +Triggers a full-data refresh or reload of specific entity types: +- Typically used for initialization or data consistency reset. + +--- + +## Examples + +### Configuring a Source Endpoint for Replication +```csharp +[ExportData(typeof(UserExposedType))] +[EventBus("Production", "https://sqs.region.amazonaws.com/account/queue")] +public class UserSourceEndpoint : SourceEndpoint { } + +var sourceEndpoint = new UserSourceEndpoint(); +sourceEndpoint.Publish(); +``` + +### Configuring a Destination Endpoint for Subscription +```csharp +public class UserDestinationEndpoint : DestinationEndpoint +{ + public UserDestinationEndpoint() : base(typeof(User).Assembly) + { + Register(typeof(User).FullName); + } +} + +// In startup or configuration method: +var destinationEndpoint = new UserDestinationEndpoint(); +await destinationEndpoint.Subscribe(); +``` + +--- + +## Configuration + +### appsettings.json Configuration Example: +```json +{ + "DataReplication": { + "MyNamespace_UserSourceEndpoint": { + "Url": "https://sqs.region.amazonaws.com/account/queue" + }, + "AllowDumpUrl": false, + "Mode": "MultiServer" + } +} +``` + +| Setting | Description | +|---------|-------------| +| **Url** | Defines the replication URL endpoint for queues. | +| **AllowDumpUrl** | Enables or disables manual data dump/refresh URLs. | +| **Mode** | Operation mode (`SingleServer` or `MultiServer`). | + +--- + +## Important Notes + +- Entity serialization/deserialization is handled via JSON (Newtonsoft.Json). +- Custom serialization behavior is available by defining CustomExposedFields or overriding serialization defaults. +- Carefully choose attributes and exposed types according to specific application replication rules. +- Handle event subscriptions properly in destination endpoints. + +--- + +## Exception Handling and Logs + +Extensive logging is provided for troubleshooting, and detailed exceptions are thrown with meaningful information regarding failed replication attempts. + +**Example exception:** +``` +Exception: Could not find a field named [FirstName] in [User] +``` \ No newline at end of file From 49af9f84b1a19c30322ab90bbbf9f6229b46a331 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 4 Mar 2025 08:17:25 +0000 Subject: [PATCH 37/68] Olive.Entities.Data.SQLite doc --- docs/Entities/Olive.Entities.Data.SQLite.md | 171 ++++++++++++++++++++ 1 file changed, 171 insertions(+) create mode 100644 docs/Entities/Olive.Entities.Data.SQLite.md diff --git a/docs/Entities/Olive.Entities.Data.SQLite.md b/docs/Entities/Olive.Entities.Data.SQLite.md new file mode 100644 index 000000000..8d409d762 --- /dev/null +++ b/docs/Entities/Olive.Entities.Data.SQLite.md @@ -0,0 +1,171 @@ +# Olive.Entities.Data.SQLite + +## Overview + +The Olive framework provides a dedicated SQLite data adapter enabling applications to efficiently store and query data using an embedded SQLite database. This adapter includes specific utilities such as generating SQL commands, handling database criteria expressions tailored for SQLite syntax, managing database lifecycle, and providing seamless integration with the Olive data access layer. + +SQLite can be a great choice particularly for mobile, desktop applications, testing, or lightweight embedded database usage scenarios. + +--- + +## Table of Contents + +- [Key Components](#key-components) +- [SqliteCommandGenerator](#sqlitecommandgenerator) +- [SqliteCriterionGenerator](#sqlitecriteriongenerator) +- [SqLiteManager](#sqlitemanager) +- [Examples](#examples) +- [Configuration](#configuration) +- [Dependencies](#dependencies) + +--- + +## Key Components + +The SQLite adapter contains the following core components: + +- **SqliteCommandGenerator:** Generates SQLite-compatible SQL commands. +- **SqliteCriterionGenerator:** Generates SQLite-specific query criteria expressions. +- **SqLiteManager:** Manages SQLite database operations and schema management. + +--- + +## SqliteCommandGenerator + +This class generates SQL command strings specifically tailored for the SQLite DBMS, fully supporting pagination and accompanying query logic. + +### Main Methods: + +- **`GenerateSelectCommand(IDatabaseQuery, string tables, string fields)`** + Constructs SQLite-compatible SELECT statements using query filters, sorting, and pagination features. + +- **`GeneratePagination(IDatabaseQuery query)`** + Implements SQLite LIMIT/OFFSET syntax to paginate query results efficiently. + +- **`SafeId(string id)`** + Safely escapes an SQLite identifier (table or column names) using backticks. + +- **`UnescapeId(string id)`** + Removes escaping from SQLite identifiers. + +**Example Usage:** +```csharp +var generator = new SqliteCommandGenerator(); +var query = new DatabaseQuery(typeof(Product)) +{ + PageStartIndex = 5, + TakeTop = 10 +}; + +string selectCommand = generator.GenerateSelectCommand(query, "`Products`", "*"); +// Result: SELECT * FROM `Products` LIMIT 10 OFFSET 5 +``` + +--- + +## SqliteCriterionGenerator + +Generates SQLite SQL fragments for entity criteria expressions such as Where clauses. It ensures identifiers follow SQLite-compatible naming and escaping. + +- **Identifier Escaping:** Uses backticks (` ` `) for safe SQLite identifier escaping. + +--- + +## SqLiteManager + +Handles SQLite database operations such as schema execution, database deletion, and command operation execution. + +### Main Methods: +- **`Delete(string databaseName)`** + Drops all tables from the SQLite database as SQLite doesn't support dropping databases directly. + +- **`Execute(string sql, string database = null)`** + Executes arbitrary SQL commands against the SQLite database. + +- **`ClearConnectionPool()`** + No operation (noop), as connection pooling is not applicable in SQLite. + +- **`Exists(string database, string filePath)`** + Currently returns always `false` as SQLite's database existence logic is file-based externally managed. + +**Important**: The `Delete` method drops all tables separately using SQLite system tables as dropping databases entirely is not supported by SQLite. + +--- + +## Examples + +### Basic SQLite Database Creation and Initialization Example: +```csharp +var manager = new SqLiteManager(providerConfig); +string createTableScript = "CREATE TABLE `Users` (`ID` TEXT PRIMARY KEY, `Name` TEXT)"; + +manager.Execute(createTableScript); +``` + +### Generating a paginated SELECT command for SQLite: + +```csharp +var query = new DatabaseQuery(typeof(User)) { PageStartIndex = 0, TakeTop = 5 }; +var commandGenerator = new SqliteCommandGenerator(); + +string command = commandGenerator.GenerateSelectCommand(query, "`Users`", "*"); +// Generated SQL: SELECT * FROM `Users` LIMIT 5 +``` + +--- + +## Configuration + +In your `appsettings.json` or configuration source, make sure to define your SQLite connection string: + +```json +{ + "ConnectionStrings": { + "Default": "Data Source=appdata.db;" + } +} +``` + +--- + +## Dependencies + +Ensure the official SQLite package is referenced in your Olive application. + +- SQLite provider via official Microsoft ADO.NET SQLite package (NuGet): + +```powershell +Install-Package Microsoft.Data.Sqlite +``` + +- Olive framework (`Olive.Entities`) NuGet package is also required: + +```powershell +Install-Package Olive.Entities +``` + +--- + +## Important Notes + +- By SQLite's nature, some standard SQL operations (dropping databases, certain ALTER operations) aren't supported or are limited. This adapter includes sensible workarounds where applicable. +- SQLite doesn't have connection pooling, so related operations in Olive's data layer are safely ignored. +- Proper identifier escaping is provided throughout Olive's SQLite adapter classes to avoid common issues with special keywords or reserved identifiers. + +--- + +## Exception Handling + +The adapter includes meaningful, Olive-specific exception messages clarifying SQL execution issues and data access errors, aiding rapid troubleshooting. + +**Example exception thrown by the manager**: + +``` +Exception: "Could not drop database 'MyDatabase'" (Inner exception details) +``` + +--- + +## Notes + +- SQLite lacks support for some database features such as multi-database setups, extensive security, and certain schema features. It is therefore recommended primarily for development, local, or lightweight applications. \ No newline at end of file From d87e994f8325ace7625e810a31a081191b2370b6 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 4 Mar 2025 08:46:17 +0000 Subject: [PATCH 38/68] Olive.Entities.Data.SqlServer doc --- .../Entities/Olive.Entities.Data.SqlServer.md | 188 ++++++++++++++++++ 1 file changed, 188 insertions(+) create mode 100644 docs/Entities/Olive.Entities.Data.SqlServer.md diff --git a/docs/Entities/Olive.Entities.Data.SqlServer.md b/docs/Entities/Olive.Entities.Data.SqlServer.md new file mode 100644 index 000000000..f35ef8c9f --- /dev/null +++ b/docs/Entities/Olive.Entities.Data.SqlServer.md @@ -0,0 +1,188 @@ +# Olive.Entities.Data.SqlServer + +## Overview + +The Olive Entities SQL Server adapter enables applications built on the Olive Framework to integrate efficiently with Microsoft SQL Server databases. It facilitates the generation and execution of optimized SQL queries specific to SQL Server's syntax and features. The adapter comprises command generators, criterion generators, database lifecycle management utilities, and simple integration extensions with Olive's Data Access infrastructure. + +--- + +## Table of Contents + +- [Key Components](#key-components) +- [SqlServerSqlCommandGenerator](#sqlserversqlcommandgenerator) +- [SqlCriterionGenerator](#sqlcriteriongenerator) +- [SqlServerManager](#sqlservermanager) +- [Example Usages](#example-usages) +- [Configuration](#configuration) +- [Dependencies](#dependencies) +- [Exception Handling and Logging](#exception-handling-and-logging) + +--- + +## Key Components + +- **SqlServerSqlCommandGenerator**: Generates SQL Server-compatible SQL queries. +- **SqlCriterionGenerator**: Creates query conditions using appropriate SQL Server escaping. +- **SqlServerManager**: Handles SQL Server-specific database operations, including deletion and detachment. + +--- + +## SqlServerSqlCommandGenerator + +Generates SQL commands tailored explicitly for SQL Server, with special consideration for pagination and sorting: + +### Primary Methods: + +- **`GenerateSelectCommand()`**: Constructs SELECT statements, optionally applying TOP, ORDER BY, and pagination clauses. +- **`GeneratePagination()`**: Uses SQL Server's `OFFSET...FETCH NEXT` pagination syntax. +- **`SafeId(string id)`**: Safely wraps SQL identifiers (columns, tables) in square brackets `[...]`. +- **`UnescapeId(string id)`**: Removes square brackets from identifiers. + +#### Example: + +```csharp +var commandGenerator = new SqlServerSqlCommandGenerator(); + +var query = new DatabaseQuery(typeof(Customer)) +{ + PageStartIndex = 20, + PageSize = 10, + OrderBy("LastName") +}; + +string sql = commandGenerator.GenerateSelectCommand(query, "[Customers]", "[ID], [FirstName], [LastName]"); + +// Generated SQL: +SELECT [ID], [FirstName], [LastName] FROM [Customers] +ORDER BY [LastName] OFFSET 20 ROWS FETCH NEXT 10 ROWS ONLY +``` + +--- + +## SqlCriterionGenerator + +Designed specifically for generating SQL Server query criteria (expressions used in WHERE clauses), using identifier escaping with square brackets: + +- **Identifier Escaping**: `[Identifier]`. +- **Identifier Unescaping**: Identifier trimmed of `[ ]`. + +--- + +## SqlServerManager + +Performs database management tasks specifically targeted at SQL Server, such as deleting or detaching databases, managing connections, and executing scripts: + +### Common Methods: + +- **`Delete(string databaseName)`**: Deletes an entire database safely by performing necessary user connection management. +- **`Detach(string databaseName)`**: Detaches the database by first disabling connections to ensure safe detachment. +- **`Execute(string sql, string database = null)`**: Executes arbitrary SQL batches. Automatically splits provided scripts on "GO" batch terminators. +- **`Exists(string database, string filePath)`**: Checks if the specified database exists in SQL Server. + +#### Example Database Deletion: + +```csharp +var manager = new SqlServerManager(); +manager.Delete("TestDatabase"); + +// Generates and executes: +// ALTER DATABASE [TestDatabase] SET SINGLE_USER WITH ROLLBACK IMMEDIATE; +// ALTER DATABASE [TestDatabase] SET MULTI_USER; +// DROP DATABASE [TestDatabase]; +``` + +--- + +## Example Usages + +### Registering SQL Server Data Access in Olive Application + +```csharp +public class Startup +{ + public void ConfigureServices(IServiceCollection services) + { + DataAccess.Configure(options => options.SqlServer()); + } +} +``` + +### Executing Scripts with SQL Server Manager + +```csharp +var manager = new SqlServerManager(); + +string sqlScript = @"CREATE TABLE [Users]( + [ID] UNIQUEIDENTIFIER PRIMARY KEY, + [Name] NVARCHAR(100) + );GO + INSERT INTO [Users] VALUES (NEWID(), 'Alice');"; + +manager.Execute(sqlScript, "MyDatabase"); +``` + +### Generating Paginated Select Command + +```csharp +var query = new DatabaseQuery(typeof(User)) { PageStartIndex = 0, PageSize = 5 }; +query.OrderBy("Name"); + +var sqlGenerator = new SqlServerSqlCommandGenerator(); +string command = sqlGenerator.GenerateSelectCommand(query, "[Users]", "[ID], [Name]"); + +// Generated SQL: +// SELECT [ID], [Name] FROM [Users] ORDER BY [Name] OFFSET 0 ROWS FETCH NEXT 5 ROWS ONLY +``` + +--- + +## Configuration + +To enable SQL Server in your Olive application, define a SQL Server connection string in your `appsettings.json` file or other configuration source: + +```json +{ + "ConnectionStrings": { + "Default": "Server=myServer;Database=myDb;User Id=myUser;Password=myPassword;" + } +} +``` +--- + +## Dependencies + +Make sure the following NuGet packages are installed and referenced in your Olive Application: + +- Official Microsoft SQL Client Provider for .NET: +```powershell +Install-Package Microsoft.Data.SqlClient +``` + +- Core Olive Libraries: +```powershell +Install-Package Olive.Entities +``` + +--- + +## Exception Handling and Logging + +The adapter provides detailed exception handling with clear, actionable error messages: + +Example Exception Thrown: +``` +Exception: "Could not drop database 'TestDB'" (Inner exceptions detailing the cause) +``` + +Logging: + +- Comprehensive logging of database operations and exceptional scenarios uses Olive's built-in logging mechanisms. +- SQL execution failures log the executed command for ease of troubleshooting. + +--- + +## Important Notes + +- Ensure that proper permissions exist for database operations executed through Olive's SQL Server adapter. +- Use square brackets `[identifier]` when dealing with identifiers that match SQL Server reserved keywords or contain spaces. +- By default, the generated SQL uses modern SQL Server pagination syntax (OFFSET...FETCH NEXT), requiring SQL Server 2012 or newer. \ No newline at end of file From 946e11883b6516270e0d171545f4cdff23ca8da8 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 4 Mar 2025 09:19:17 +0000 Subject: [PATCH 39/68] Olive.Export doc --- docs/Services/Olive.Export.md | 208 ++++++++++++++++++++++++++++++++++ 1 file changed, 208 insertions(+) create mode 100644 docs/Services/Olive.Export.md diff --git a/docs/Services/Olive.Export.md b/docs/Services/Olive.Export.md new file mode 100644 index 000000000..7ff3c7def --- /dev/null +++ b/docs/Services/Olive.Export.md @@ -0,0 +1,208 @@ +# Olive.Export + +## Overview +The Olive Data Export module provides flexible utilities for exporting structured data into popular formats such as Excel XML and CSV. It includes robust customization mechanisms allowing fine-grained control over data content, column configuration, styling, and specialized functionalities like formulas, dropdowns (enumerations), hyperlinks, and grouped columns. + +## Table of Contents +- [Key Components](#key-components) +- [Classes and Their Usage](#classes-and-their-usage) + - [Exporter](#exporter) + - [Exporter<T>](#exporter-t) + - [Column<T>](#column-t) + - [DropDownColumn<T>](#dropdowncolumn-t) + - [Cell](#cell-class) + - [CellStyle](#cellstyle-class) +- [Supported Export Formats](#supported-export-formats) +- [Usage Examples](#usage-examples) +- [Generating Excel XML](#generating-excel-xml) +- [Generating CSV](#generating-csv) +- [Advanced Styling Examples](#advanced-styling-examples) +- [Dependencies](#dependencies) +- [Exception Handling and Logging](#exception-handling-and-logging) +- [Important Notes](#important-notes) + +--- + +## Key Components +- **Exporter classes** (`Exporter`, `Exporter`) for initiating and managing the export process. +- **Column definitions** (`Column`, `DropDownColumn`) to structure and customize data fields. +- **Cells** (`Cell`) encapsulating data and their individual styling. +- **Styling system** (`CellStyle`) to control visual presentation including fonts, colors, alignment, and borders. +- **Multiple file formats** support (`ExcelXml`, `CSV`) for versatile data exchange. + +--- + +## Classes and Their Usage + +### Exporter +The base exporter class (`Exporter`) facilitates exporting arbitrary data: +- Can be directly initialized with a `System.Data.DataTable`. +- Provides methods and enums for alignment and orientation customization. + +### Exporter\ +Typed exporter for strongly-typed collections of objects: +- Manages document name, columns, rows, and sheet-level settings. +- Allows easy definition of custom columns and styling. + +### Column\ +Represents specific columns for export spreadsheets. Supports extensive customization: +- **HeaderText**: Column title. +- **DataType**: Data format (e.g., String, Number). +- **Style management**: Custom styles for headers/rows. +- **Formulas**: Excel formulas for automated calculations. +- **Grouping**: Supports grouped column headers. + +### DropDownColumn\ +Inherits from `Column`. Provides Excel dropdown functionality: +- **PossibleValues** array for setting possible dropdown selections. +- **EnumerationName**: Named range in Excel to bind the cell dropdown. + +### Cell Class +Represents individual spreadsheet cells holding data and styling. +- **Text and Type** attributes to define content and data type. +- Style customization using the embedded `CellStyle` object. + +### CellStyle Class +Customization of cell appearance with granular settings such as alignment, fonts, colors, borders, rotations (`Vertical/Horizontal` orientations), background, etc. + +--- + +## Supported Export Formats +- **Excel XML** (`.xls`): A versatile spreadsheet format compatible with Microsoft Excel. +- **Comma Separated Values (CSV)** (`.csv`): A simple text-based data format usable in many applications. + +--- + +## Usage Examples + +### Exporting Basic Data to Excel +```csharp +var exporter = new Exporter("BasicExport"); +exporter.AddColumn("Name"); +exporter.AddColumn("Age", "Number"); +exporter.AddRow("Alice", 30); +exporter.AddRow("Bob", 25); +var document = exporter.ToDocument(Format.ExcelXml); +``` + +### Exporting a List with a Generic Exporter +```csharp +var items = new List { employee1, employee2 }; + +var exporter = new Exporter("Employees Report"); +exporter.AddColumn("Name", "String", e => e.Name); +exporter.AddColumn("Hire Date", "DateTime", e => e.HireDate); + +exporter.AddRows(items); +var result = exporter.Generate(Format.ExcelXml); +``` + +### DropDown Column Definition +```csharp +var exporter = new Exporter("DropdownFile"); +exporter.AddDropDownColumn("Department", "String", "Departments", new[] {"IT","HR","Sales"}); +exporter.AddRow("IT"); +``` + +--- + +## Generating Excel XML +Supports advanced Excel features: +- Column freezing with `FreezeHeader` options. +- Cell merge functionality for header groupings. +- Links and formula support. +- Drop-down validations via enumerations. + +### Example Excel XML generation +```csharp +var exporter = new Exporter("Report"); +exporter.AddColumn("Name"); +exporter.AddColumn("Salary", "Number").SetRowStyle(s => s.NumberFormat = "#,##0.00"); +exporter.AddRow("John", 65000); +var xml = exporter.Generate(Format.ExcelXml); +``` + +--- + +## Generating CSV +```csharp +var exporter = new Exporter("SimpleCSV"); +exporter.AddColumn("Product"); +exporter.AddColumn("Price", "Number"); +exporter.AddRow("Laptop", 1000); +exporter.AddRow("Phone", 800); +var csv = exporter.Generate(Format.Csv); +``` + +--- + +## Custom Styling with CellStyle +```csharp +var exporter = new Exporter("StyledDocument"); +var column = exporter.AddColumn("Name"); +column.HeaderStyle.FontSize = 14; +column.HeaderStyle.Bold = true; +column.SetRowStyle(s => { + s.ForeColor = "#003366"; + s.BackgroundColor = "#EEEEEE"; +}); + +exporter.AddRow("Chris"); +exporter.AddRow("Alex"); +``` + +--- + +## Dependencies +Install the following NuGet packages to leverage Olive Export features: + +- Olive Framework: +```powershell +Install-Package Olive.Entities +``` + +- Olive Core Framework (`Olive`): +```powershell +Install-Package Olive +``` + +--- + +## Exception Handling and Logging +The exporter includes descriptive exceptions for troubleshooting: +- Validation errors (e.g., mismatched column counts). +- Invalid style settings (e.g., invalid border width). +- Errors in Excel formulas or CSV generation. + +Detailed logging can be configured using Olive's built-in logging features, providing detailed insights into problematic export operations. + +**Example**: +``` +Exception: Invalid Link value for ExportToExcel: [provided invalid link string] +``` + +--- + +## Important Notes +- An Excel XML file generated with this exporter is XML-based and supports basic Excel features. Excel files generated as `.xls` from XML format might show a warning when opening in Excel; this is standard behavior due to Microsoft Excel's security model. +- Make sure sheet names used in one Excel workbook are unique. +- The current implementation handles standard string, numeric, and link data types. Extend carefully for additional complex scenarios. + +--- + +## Dependencies +- `Olive.Entities` (core Olive data framework) +- `Olive` Core Extensions for additional utility methods + +Include via NuGet: +```powershell +Install-Package Olive.Entities +Install-Package Olive +``` + +--- + +## Important Notes +- Remember to escape special CSV characters using the provided Olive extension methods. +- Excel XML output supports style-rich output, while CSV is limited to plain text. +- Avoid excessively large datasets in Excel XML, as this might lead to slower opening or increased memory usage. For large datasets, consider CSV exports. \ No newline at end of file From b1f5a4b5b4d7de69a80a8a470c82d4898ab2372a Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 4 Mar 2025 10:03:44 +0000 Subject: [PATCH 40/68] Olive.GeoLocation doc --- docs/Services/Olive.GeoLocation.md | 261 +++++++++++++++++++++++++++++ 1 file changed, 261 insertions(+) create mode 100644 docs/Services/Olive.GeoLocation.md diff --git a/docs/Services/Olive.GeoLocation.md b/docs/Services/Olive.GeoLocation.md new file mode 100644 index 000000000..fb5ab8e4c --- /dev/null +++ b/docs/Services/Olive.GeoLocation.md @@ -0,0 +1,261 @@ +# Olive GeoLocation Documentation + +## Overview + +Olive GeoLocation provides robust functionality for handling geographical data and calculations. It simplifies operations such as retrieving geographic coordinates, calculating distances between locations, and working with different geographic regions. Leveraging Olive's integration, it seamlessly interacts with Google Maps APIs for accurate geospatial calculations. + +--- + +## Table of Contents + +- [Key Functionalities](#key-functionalities) +- [Installation and Setup](#getting-started) +- [Core Components](#core-components) + - [GeoLocation Class](#geolocation-class) + - [Distance Struct](#distance-struct) + - [GeoLocationService](#geolocationservice) + - [RadialRegion and RectangularRegion](#radialregion-and-rectangularregion-classes) +- [Configuration](#configuration) +- [Usage Examples](#usage-examples) +- [Dependencies](#dependencies) +- [Exception Handling and Logging](#exception-handling-and-logging) +- [Important Notes](#important-notes) + +--- + +## Key Functionalities + +- **Determining Geo Coordinates:** Get longitude and latitude for locations and postcodes using Google's Geocoding API. +- **Distance Calculations:** Compute geographic distances (straight-line) and travel distances between points. +- **Geographical Regions:** Defining and using specific rectangular and radial regions for geographic analyses. + +--- + +## Getting started + +First, install the [Olive.GeoLocation](https://www.nuget.org/packages/Olive.GeoLocation/) NuGet package into your project: + +``` +Install-Package Olive.GeoLocation +``` + +Then register the GeoLocation services using dependency injection in your application (`Startup.cs` file): + +```csharp +public override void ConfigureServices(IServiceCollection services) +{ + base.ConfigureServices(services); + services.AddGeoLocationService(); +} +``` + +--- + +## Key Functionalities + +### GeoLocation Class + +The core representation of geographic points with latitude and longitude coordinates. + +Properties: +- **Latitude** (`double`) — Latitude coordinate. +- **Longitude** (`double`) — Longitude coordinate. + +### Parse Example: + +```csharp +var location = GeoLocation.Parse("51.5074,-0.1278"); +Console.WriteLine(location.Latitude); // Outputs: 51.5074 +``` + +--- + +### Distance Struct + +Represents distances and provides convenient conversions between meters, kilometers, and miles. + +Properties: +- `Meters` — Total meters. +- `Kilometers` — Converted from meters. +- `Miles` — Converted from meters to miles. + +Constructors: +- `FromMeters(double meters)` +- `FromMiles(double miles)` +- `FromKilometers(double kilometers)` + +### Example Usage: + +```csharp +var distance = Distance.FromMiles(10); +Console.WriteLine($"Kilometers: {distance.Kilometers} km"); // Outputs: "Kilometers: 16.093 km" +``` + +--- + +## RadialRegion and RectangularRegion Classes + +Define geospatial regions used in geographical calculations or analyses: + +### RadialRegion + +Represents a radial (circular) geographic area defined by a central point and radius. + +### RectangularRegion + +Represents rectangular geographic areas by two opposite corners. + +--- + +## Configuration + +After installing the package, ensure you configure the Google Maps API integration properly by adding the following to your `appsettings.json` file: + +```json +{ + "Google": { + "Maps": { + "Api": { + "Client": { + "Key": "[YOUR_GOOGLE_API_KEY]" + }, + "Signature": "[YOUR_GOOGLE_API_SIGNATURE]" + } + } + } +} +``` + +_**Important**_: Obtain your API key from the [Google Cloud Console](https://console.cloud.google.com/). + +--- + +## Using the GeoLocation Service + +The GeoLocation service is configured via dependency injection and can be used to perform various geolocation calculations: + +```csharp +public class ActionRequest +{ + readonly IGeoLocationService GeoLocationService; + + public ActionRequest(IGeoLocationService geoLocationService) + { + GeoLocationService = geoLocationService; + } +} +``` + +### Methods available: + +- **GetPostcodeLocation**: Retrieves latitude and longitude for a specified postcode. +- **CalculateDistance**: Calculates the straight-line distance (geographical miles) between two postcodes. +- **CalculateTravelDistance**: Computes actual travel distance between locations using Google Maps API. + +--- + +## RadialRegion and RectangularRegion Classes + +### RadialRegion + +Defines a radial (circular) geographic area based on a center and radius. + +```csharp +var center = new GeoLocation { Latitude = 51.5074, Longitude = -0.1278 }; +var radialRegion = new RadialRegion(center, Distance.FromKilometers(10)); +``` + +### RectangularRegion + +Defines a rectangular area by specifying top-left and bottom-right coordinates. + +```csharp +var region = new RectangularRegion +{ + TopLeft = new GeoLocation { Latitude = 50.0, Longitude = -10.0 }, + BottomRight = new GeoLocation { Latitude = 40.0, Longitude = 0.0 } +}; + +var midpoint = region.Centre(); // Center of the rectangle +``` + +--- + +## Configuration + +Ensure these app configuration settings exist: + +```json +{ + "Google": { + "Maps": { + "Api": { + "Client": { + "Key": "YOUR_API_KEY" + }, + "Signature": "[Optional_Google_Signature]" + } + } + } +} +``` + +--- + +## Usage Examples + +### Calculating Geo-distance Between Locations + +```csharp +var geo = new GeoLocationService(); + +var location1 = await geo.GetPostcodeLocation("E14 5AB"); +var location2 = await geoLocationService.GetPostcodeLocation("W1A 1AA"); + +var distance = location1.GetDistance(location2); // Distance in miles +``` + +### Calculate Travel Distance (Route) Using Google Maps API: + +```csharp +var travelDistance = await geoLocationService.CalculateTravelDistance("E14 5AB", "W1A 1AA"); +Console.WriteLine($"Travel distance: {travelDistance} miles"); +``` + +--- + +## Exception Handling and Logging + +The GeoLocation library provides detailed error handling and logging mechanisms, including: + +- Validation errors for invalid GeoLocation data +- Comprehensive exceptions detailing Google API errors +- Logs configuration mistakes, such as missing or invalid API keys + +**Example logged exception:** + +```text +Exception: Google API Error: REQUEST_DENIED +``` + +--- + +## Dependencies + +- Olive core library and entities packages required: + +```bash +Install-Package Olive +Install-Package Olive.Entities +``` + +- Configured Google Maps API key (with GEOCODE API access enabled). + +--- + +## Important Notes + +- Google API has usage limits (default daily limit of around 25,000 requests) unless extended with billing enabled. +- Verify your Google API quotas and monitor usage to prevent exceeding daily limits. +- GeoLocation representations might slightly vary depending on region data accuracy. +- Ensure security of geographic data (e.g., anonymized or authorized access only). \ No newline at end of file From 317527ff976249cdb5624cc76b2039d34b239563 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 4 Mar 2025 10:35:18 +0000 Subject: [PATCH 41/68] Olive.Globalization doc --- docs/Services/Globalization.md | 243 ++++++++++++++++++++++++++++++++- 1 file changed, 242 insertions(+), 1 deletion(-) diff --git a/docs/Services/Globalization.md b/docs/Services/Globalization.md index b4cef4cb5..c1a386224 100644 --- a/docs/Services/Globalization.md +++ b/docs/Services/Globalization.md @@ -1 +1,242 @@ -# Olive.Globalization \ No newline at end of file +# Olive.Globalization + +## Overview + +The Olive Globalization module provides comprehensive language localization and translation capabilities, making it easy to support multiple languages within Olive-based applications. It seamlessly integrates translation services such as Google Translate, caches translations locally to improve performance, and provides utilities to translate strings, phrases, and entire HTML contents efficiently. + +--- + +## Table of Contents + +- [Getting Started](#getting-started) + - [Installation](#installation) + - [Configuration](#configuration) +- [Key Features](#key-features) + - [Translation Utilities](#translation-utilities) + - [Integrating Google Translate](#integrating-google-translate) + - [Automatic Translation Caching](#automatic-translation-caching) + - [Language and Phrase Interfaces](#language-and-phrase-interfaces) +- [Core Components](#core-components) + - [Translator](#translator-class) + - [ILanguage Interface](#ilanguage-interface) + - [ITranslationProvider Interface](#itranslationprovider-interface) + - [IPhraseTranslation Interface](#iphrasetranslation-interface) +- [Using EnglishSpelling Helper](#using-englishspelling-helper) +- [Usage Examples](#usage-examples) +- [Handling HTML Translations](#handling-html-translations) +- [Customization and Advanced Use](#customization-and-advanced-use) +- [Dependencies](#dependencies) +- [Exception Handling and Logging](#exception-handling-and-logging) +- [Important Notes](#important-notes) + +--- + +## Getting Started + +### Installation + +Include Olive Globalization in your project via NuGet: + +```powershell +Install-Package Olive.Globalization +``` + +### Configuration + +Configure Google Translate API keys in your `appsettings.json` file: + +```json +{ + "Globalization": { + "GoogleTranslate": { + "Enabled": true, + "ApiKey": "[YOUR_GOOGLE_TRANSLATE_API_KEY]", + "QueryLength": 115, + "CharacterLimit": 2000 + } + } +} +``` + +--- + +## Key Features + +### Translation Utilities + +Provides seamless translation of phrases with support for special character preservation and decorator handling. + +- Easy-to-use API methods +- Automatic support for determining context language and default language +- Prevents unnecessary translation API calls with local caching + +### Integrating Google Translate + +- Supports Google's powerful translation services +- Caches translations locally to optimize cost and improve performance +- Can detect the language automatically and provide confidence levels + +### Automatic Translation Caching + +Translations from external translation providers (such as Google Translate) are automatically cached locally, significantly improving translation performance and reducing external API calls. + +### Language and Phrase Interfaces + +Abstraction through interfaces (`ILanguage`, `IPhraseTranslation`, and `ITranslationProvider`) enables easy integration and customization of translation and localization services. + +--- + +## Core Components + +### Translator Class + +The main class managing translations and coordinating between local cache and external providers. + +**Methods & Events:** + +- `Translate(...)`: Translate plain text or phrases to the desired language. +- `TranslateHtml(...)`: Translate entire HTML content intelligently while preserving its structure. +- `TranslationDownloaded` event: Invoked when a translation is downloaded externally. + +### ILanguage Interface + +Provides language definitions, such as English, French or Spanish. + +**Properties:** + +- `string Name`: Friendly name (e.g. "English"). +- `string IsoCode`: ISO language code (e.g. "en"). +- `bool IsDefault`: Determines if this is the application's default language. + +### ITranslationProvider Interface + +Defines methods for external translation providers that can translate phrases to different languages. + +Methods: + +- `Translate(string phrase, string sourceIsoCode, string targetIsoCode)`: Translate phrases between languages. +- `MaximumTextLength`: Maximum characters supported per translation request. + +### IPhraseTranslation Interface + +Defines translations fetched from translation providers, stored locally for reuse. + +Properties: + +- `Phrase`: Original phrase +- `Translation`: Translated text +- `Language`: Language to which phrase was translated + +--- + +## Using EnglishSpelling Helper + +Olive provides functionality to switch between British and American spelling variants of English content. + +Usage Example: + +```csharp +var britishText = EnglishSpelling.ToBritish("color"); +var americanText = EnglishSpelling.ToAmerican("colour"); +``` + +--- + +## Usage Examples + +### Translating Simple Text Phrases + +```csharp +// Translate "Hello World" into Spanish +var language = await Context.Current.Language(); +var translation = await Translator.Translate("Hello World", language); +``` + +### Implementing TranslationDownloaded Event Handler + +```csharp +Translator.TranslationDownloaded += async args => +{ + // Save translations locally when downloaded + await Database.Save(entity); +}; +``` + +--- + +## Handling HTML Translations + +Efficiently translates complex HTML content structures without breaking their formatting. + +```csharp +var originalHtml = "

Hello World

"; +var language = await Context.Current.Language(); + +var translatedHtml = await Translator.TranslateHtml(originalHtml, language); +``` + +--- + +## Customization and Advanced Use + +### Implementing Custom Context Language Provider + +Implement `IContextLanguageProvider` to define a custom logic to retrieve the current language of a context: + +```csharp +public class MyLanguageProvider : IContextLanguageProvider +{ + public Task GetCurrentLanguage() + { + // Get language based on custom logic + } +} +``` + +Register this provider in your application's dependency injection container. + +### Handling TranslationDownloaded Events + +You can hook into global translation events. Useful for logging or storing translations: + +```csharp +Translator.TranslationDownloaded += async args => +{ + // Track downloaded translations + await Database.Save(entity); +}; +``` + +--- + +## Dependencies + +- Olive framework core packages: +``` +Install-Package Olive +Install-Package Olive.Entities +Install-Package Olive.Globalization +``` + +- Google API Access: Configure Google Translate API keys properly before using the service. + +--- + +## Exception Handling and Logging + +- Comprehensive logging of translation process via Olive's built-in logging. +- Detailed exception handling and logging for translation API errors. +- Informs of configuration issues (e.g., invalid API keys or exceeded quotas). + +Example exception message: + +``` +Exception: Google API Error: REQUEST_DENIED +``` + +--- + +## Important Notes + +- Google Translate has a query character limit per request. Ensure text chunks do not exceed configured limit. +- Regularly monitor translation quotas and API usage limits provided by Google. \ No newline at end of file From 0e8426157860a8e54db69a139f880d15cc5a3244 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 4 Mar 2025 11:02:00 +0000 Subject: [PATCH 42/68] Olive.Hangfire.Cron doc --- docs/Services/Olive.Hangfire.Cron.md | 95 ++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 docs/Services/Olive.Hangfire.Cron.md diff --git a/docs/Services/Olive.Hangfire.Cron.md b/docs/Services/Olive.Hangfire.Cron.md new file mode 100644 index 000000000..07f134219 --- /dev/null +++ b/docs/Services/Olive.Hangfire.Cron.md @@ -0,0 +1,95 @@ +# Olive.Hangfire.Cron (Hangfire Integration) + +## Overview + +The Olive.Hangfire.Cron includes convenient scheduling utilities leveraging the standard cron expression syntax via the `Cron` class, inspired by the Hangfire task scheduling library. This class provides simple methods to generate cron expressions which represent common recurring time intervals, simplifying scheduler configurations for background tasks in applications. + +--- + +## Table of Contents + +- [Overview](#overview) +- [Cron Class Functionalities](#cron-class-functionalities) +- [Supported Scheduling Methods](#supported-scheduling-methods) +- [Method Usage Examples](#method-usage-examples) +- [Exception Handling](#exception-handling) + +--- + +## Cron Class Functionalities + +The `Cron` helper class contains predefined templates and methods for generating cron expressions compatible with the popular Hangfire job scheduler. It includes methods for creating cron expressions easily for common time intervals such as hourly, daily, weekly, monthly, and yearly. + +Each method returns a cron string in the well-known "* * * * *" syntax, indicating the exact interval or frequency for scheduled tasks. + +--- + +## Supported Scheduling Methods + +The available methods in this utility class are: + +| Method | Description | Example cron format | +| ------------------- | -----------------------------------|-----------------------| +| `Never()` | Returns an expression that never fires (31st February). | `"0 0 31 2 *"` | +| `Hourly(int minute)` | Fires every hour at the specified minute | +| `Daily(int hour, int minute)` | Fires every day at a specified hour and minute | +| `Weekly(DayOfWeek dayOfWeek, int hour, int minute)` | Fires weekly at the given day and time | +| `Monthly(int day, int hour, int minute)` | Fires monthly at the provided day and time | +| `Yearly(int month, int day, int hour, int minute)` | Fires annually at the specific month, day, and time | +| `MinuteInterval(int interval)` | Fires every "specified interval" minutes | +| `HourInterval(int interval)` | Fires every specified number of hours | +| `MonthInterval(int interval)` | Fires every specified number of months | +| `Never()` | Cron expression that never fires | + +--- + +## Method Usage Examples + +### Scheduling tasks hourly: +```csharp +// Schedule at minute 15 of every hour +var hourlyCron = Cron.Hourly(15); +// hourlyCron value is: "15 * * * *" +``` + +### Scheduling tasks daily: +```csharp +// Every day at 9:30 AM +var dailyCron = Cron.Daily(9, 30); +``` + +### Scheduling tasks weekly: +```csharp +// Every Friday at 18:00 +var weeklyCron = Cron.Weekly(DayOfWeek.Friday, 18, 0); +``` + +### Scheduling tasks monthly: +```csharp +// Every month on the 1st day, 9 AM and 30 minutes: +var monthlyCron = Cron.Monthly(1, 9, 30); +``` + +### Scheduling a task every n months: +```csharp +// Fires on 1st day every 3 months at 00:00 +var everyThreeMonthsCron = Cron.MonthInterval(3); +``` + +### Scheduling yearly: +```csharp +// Every year on March 10th at midnight +var yearlyCron = Cron.Yearly(month:3, day:10); +``` + +--- + +## Exception Handling and Logging + +- Olive's Cron methods include built-in validation for parameter values (e.g., minutes and hours are checked to ensure valid cron generation). +- Each method throws meaningful exceptions if provided with invalid parameter values outside logical intervals. + +Example: +```csharp +var cronExpression = Cron.Hourly(70); // Throws ArgumentOutOfRangeException +``` \ No newline at end of file From e81d62a85e41226eb99b3777a214ab929fa60fc7 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 4 Mar 2025 11:15:51 +0000 Subject: [PATCH 43/68] Olive.Hangfire.MySql doc --- docs/Services/Olive.Hangfire.MySql.md | 135 ++++++++++++++++++++++++++ 1 file changed, 135 insertions(+) create mode 100644 docs/Services/Olive.Hangfire.MySql.md diff --git a/docs/Services/Olive.Hangfire.MySql.md b/docs/Services/Olive.Hangfire.MySql.md new file mode 100644 index 000000000..49a5f5ee4 --- /dev/null +++ b/docs/Services/Olive.Hangfire.MySql.md @@ -0,0 +1,135 @@ +# Olive.Hangfire.MySql - Hangfire Integration (MySQL) + +## Overview + +This Olive.Hangfire.MySql integrates Olive applications with Hangfire to provide scheduled background job processing. It enables your Olive application to easily leverage recurring task execution using Hangfire's powerful scheduling capabilities backed specifically by MySQL storage. It simplifies the configuration and registration of scheduled tasks, background job processing, and monitoring via Hangfire dashboard. + +--- + +## Table of Contents + +- [Getting Started](#getting-started) +- [Configuration](#configuration) +- [Core Extension Methods](#core-extension-methods) + - [`AddScheduledTasks`](#addscheduledtasks) +- [Monitoring Jobs with Dashboard](#enabling-hangfire-dashboard) +- [Exception Handling](#exception-handling) +- [Dependencies](#dependencies) +- [Important Notes](#important-notes) + +--- + +## Getting Started + +To use Olive Scheduled Tasks with MySQL through Hangfire, install the required NuGet packages: + +```powershell +Install-Package Hangfire +Install-Package Hangfire.MySql +``` + +Then register the scheduled tasks in your application's `Startup.cs` file. + +**Step 1: Configure in `Startup.cs` (inside `ConfigureServices`)** + +```csharp +services.AddScheduledTasks(); // Adds basic scheduled tasks capability. +``` + +--- + +## Configuration + +Specify your connection settings within `appsettings.json`: + +```json +{ + "ConnectionStrings": { + "Default": "server=myserver;database=mydatabase;uid=user;pwd=password;Allow User Variables=True;" + } +} +``` + +Ensure this default connection string points to your MySQL database, where Hangfire data will be stored. + +--- + +## Core Functionalities + +- **Easy Setup:** Quickly configure scheduled tasks and Hangfire integration for MySQL. +- **Integrated Dashboard:** Automatically starts Hangfire dashboard when debugging, granting visual monitoring of jobs. +- **Recurring Task Registration:** Use BackgroundJobsPlans to clearly define your periodic tasks. + +--- + +## Core Methods + +### AddScheduledTasks (IServiceCollection Extension) + +Registers Hangfire and initializes it with MySQL-backed storage using Olive's current database connection string. + +```csharp +services.AddScheduledTasks(); +``` + +--- + +### UseScheduledTasks\ (IApplicationBuilder Extension) + +Initializes Hangfire server to execute the scheduled tasks defined in your project. When debugging, automatically starts Hangfire dashboard on a convenient URL. + +**Example Usage:** + +```csharp +app.UseScheduledTasks(); +``` + +Here `BackgroundJobsPlan` is a user-defined plan class listing recurrent tasks. + +--- + +## Integration Details + +The Olive integration configures Hangfire to use MySQL as its job storage backend, specifically utilizing Hangfire.MySql. This is optimized for applications already using MySQL as their primary database, providing seamless integration and reduced setup complexity. + +--- + +## Exception Handling + +Meaningful exceptions and errors are managed by the Olive integration: + +- Database connection issues are raised clearly during startup: +``` +Exception: "Failed to open a DB connection." +``` + +- Misconfigured or invalid cron expressions also raise exceptions. + +Additional logs and exceptions related to missing configurations or connection issues clearly describe failing situations: + +``` +"Could not connect to the provided MySQL database." +``` + +--- + +## Dependencies + +Ensure the following NuGet packages are installed and configured: + +```powershell +Install-Package Hangfire +Install-Package Hangfire.MySql +Install-Package Olive +Install-Package Olive.Entities +``` + +Ensure `MySql.Data` or `MySqlConnector` is also installed for MySQL connectivity. + +--- + +## Important Notes + +- Verify application permissions for creating and managing MySQL schema needed by Hangfire. +- Ensure that production systems are secured, and restrict Hangfire dashboard access appropriately. +- Hangfire's MySQL storage requires a properly configured database, including correct character encoding support (utf8mb4_unicode_ci recommended) to avoid issues such as truncation. \ No newline at end of file From ecfd348784cef3d3f5ec0747f9eb370997aead6e Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 4 Mar 2025 12:02:57 +0000 Subject: [PATCH 44/68] Olive.Hangfire doc --- docs/Services/Olive.Hangfire.md | 127 ++++++++++++++++++++++++++++++++ 1 file changed, 127 insertions(+) create mode 100644 docs/Services/Olive.Hangfire.md diff --git a/docs/Services/Olive.Hangfire.md b/docs/Services/Olive.Hangfire.md new file mode 100644 index 000000000..415f37681 --- /dev/null +++ b/docs/Services/Olive.Hangfire.md @@ -0,0 +1,127 @@ +# Olive.Hangfire + +## Overview + +The Olive Scheduled Tasks library allows Olive applications to easily integrate scheduled-background tasks execution using **Hangfire**, specifically configured to leverage SQL Server storage for reliable, distributed task scheduling. Additionally, it provides developer utilities for debugging and running recurring tasks manually. + +--- + +## Table of contents + +- [Features](#key-features) +- [Getting Started](#getting-started) +- [Core Classes and Methods](#core-classes-and-methods) +- [Configuration](#configuration) +- [Dependencies](#dependencies) +- [Logging and Exception Handling](#logging-and-exception-handling) +- [Notes](#important-notes) + +--- + +## Key Features + +- Quickly integrate scheduled tasks in ASP.NET Core applications using Hangfire. +- Supports both SQL Server and MySQL storage providers via extension methods. +- Automatic task scheduling configuration via managed cron expressions. +- Developer-friendly tasks dashboard integration (Hangfire's built-in dashboard). +- Provides developer commands for debugging task executions directly from the browser UI. +- Seamless integration with Olive's Data infrastructure. + +--- + +## Getting Started + +Install the required NuGet packages: + +```powershell +Install-Package Hangfire +Install-Package Hangfire.SqlServer +Install-Package Olive +Install-Package Olive.Entities +``` + +--- + +## Core Classes and Methods + +### `SqlServerStorage` and Middleware Initialization + +The Olive solution integrates Hangfire using SQL Server as the storage backend for scheduled tasks. + +### Extensions for .NET Core ease-of-use (`AddScheduledTasks`) + +- Extension methods simplify configuring Hangfire with Olive integration in the ServiceCollection: + +```csharp +services.AddScheduledTasks(); +``` + +--- + +## Configuration (`appsettings.json`): + +```json +{ + "ConnectionStrings": { + "Default": "Server=myServer;Database=myDB;Trusted_Connection=True;" + }, + "Automated.Tasks": { + "Enabled": true + } +} +``` + +--- + +## Developer Tools: `ScheduledTasksDevCommand` + +Olive offers developer utilities to selectively run scheduled tasks manually, useful in development and debugging scenarios: + +- Easy manual invocation through provided developer commands. +- Visual aids to see tasks clearly in a browser as HTML table views. + +Example Developer Command usage via browser: + +``` +http://yourwebsite-domain/cmd/scheduled-tasks?run=DailyCleanupJob +``` + +--- + +## Dependencies + +You must install and configure these packages and libraries: + +- **Hangfire**: Scheduling engine. +- **Hangfire.SqlServer**: Storage provider for Microsoft SQL Server. +- **Olive Framework**: Provides integrated Context and Database access. + +Install via NuGet: + +```powershell +Install-Package Hangfire +Install-Package Hangfire.SqlServer +Install-Package Olive +Install-Package Olive.Entities +``` + +--- + +## Logging and Exception Handling + +- Comprehensive logging for scheduled task executions and failures. +- Robust exceptions clearly identifying runtime errors or misconfigurations, e.g., missing database configuration. + +Example Exception message: + +``` +Exception: Failed to open a DB connection. ConnectionString was invalid or database unreachable. +``` + +--- + +## Important Notes + +- Ensure reliable database connectivity and permissions since job execution history and scheduling is stored in SQL Server. +- Consider securing your Hangfire dashboard for production deployments when Environments aren't development or debugging mode. +- Validate cron expressions and scheduled task execution carefully before enabling automated tasks in production. \ No newline at end of file From 5dd08c9b5e1c95e575541ad001da8e32e2c4f5f9 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 4 Mar 2025 12:24:17 +0000 Subject: [PATCH 45/68] Olive.Log.EventBus doc --- docs/Distributed/Olive.Log.EventBus.md | 191 +++++++++++++++++++++++++ 1 file changed, 191 insertions(+) create mode 100644 docs/Distributed/Olive.Log.EventBus.md diff --git a/docs/Distributed/Olive.Log.EventBus.md b/docs/Distributed/Olive.Log.EventBus.md new file mode 100644 index 000000000..a4cf1bb9c --- /dev/null +++ b/docs/Distributed/Olive.Log.EventBus.md @@ -0,0 +1,191 @@ +# Olive.Log.EventBus + +## Overview + +**Olive.Log.EventBus** is a robust logging implementation tailored for Olive-based microservices, providing infrastructure to seamlessly publish log events onto an Event Bus via message queues. This mechanism enables centralized log collection, real-time monitoring, and easy integration with various logging and monitoring services, ensuring comprehensive, scalable, and unified logging across distributed Olive applications. + +--- + +## Table of Contents + +- [Overview](#overview) +- [Features](#features) +- [Installation](#installation) +- [Configuration](#configuration) +- [Core Components](#core-components) + - [`EventBusLogger`](#eventbuslogger-class) + - [`EventBusLoggerProvider`](#eventbusloggerprovider-class) + - [`EventBusLoggerOptions`](#eventbusloggeroptions-class) + - [`EventBusLoggerMessage`](#eventbusloggermessage-class) +- [Usage Examples](#usage-examples) +- [Exception Handling](#exception-handling) +- [Dependencies](#dependencies) +- [Important Notes](#important-notes) + +--- + +## Features + +- **Distributed Logging**: Publish log entries directly to an event bus message queue. +- **Batching Support**: Reduces overhead by sending multiple log entries in batches. +- **Centralized Log Collection**: Facilitates easier monitoring and management of logs across multiple microservices. +- **Easy Integration**: Seamless integration into ASP.NET Core logging pipeline using existing Microsoft Logging APIs. +- **Detailed Exception Handling**: Advanced logging of exceptions including stack trace and custom details. + +--- + +## Installation + +To enable EventBus Logging in your Olive application, ensure these NuGet packages are installed: + +```powershell +Install-Package Olive +Install-Package Olive.Logging +Install-Package Microsoft.Extensions.Logging +``` + +--- + +## Configuration + +Configure EventBus Logging in your `appsettings.json`: + +```json +{ + "Logging": { + "EventBus": { + "QueueUrl": "[YOUR_EVENT_BUS_QUEUE_URL]", + "Source": "[YOUR_MICROSERVICE_NAME]" + } + } +} +``` + +Alternatively, configure via code in your application's startup (`Program.cs` or `Startup.cs`): + +```csharp +builder.Logging.AddEventBus(options => +{ + options.QueueUrl = "[YOUR_EVENT_BUS_QUEUE_URL]"; + options.Source = "[YOUR_MICROSERVICE_NAME]"; +}); +``` + +--- + +## Core Components + +### `EventBusLogger` Class + +Custom logger implementation extending Olive's batching logging framework. + +- **Core method**: `Log(...)` generates log entries forwarded to the event bus. + +### `EventBusLoggerProvider` Class + +Provides and manages `EventBusLogger` instances and handles sending batched log entries via the Event Bus. + +- **Initialization**: Fetches QueueUrl and Source from options or configuration. +- **Method**: `WriteMessagesAsync()` sends batched logs to the configured event bus. + +### `EventBusLoggerOptions` Class + +Holds configuration options necessary for `EventBusLoggerProvider`. + +| Property | Description | +|----------|-------------| +| `QueueUrl` | The URL for the message queue (EventBus) | +| `Source` | Identifies the source microservice or application | + +### `EventBusLoggerMessage` Class + +Data structure used to send log messages across the event bus. + +| Property | Description | +|----------|-------------| +| `Messages` | A collection of log events encapsulated in this batch | +| `Date` | Timestamp of the log batch creation | +| `Source` | The originating service or application | + +--- + +## Usage Examples + +### Logging Information + +```csharp +public class OrderService +{ + readonly ILogger Logger; + + public OrderService(ILogger logger) + { + Logger = logger; + } + + public void ProcessOrder() + { + Logger.LogInformation("Processing a new order."); + // ... + } +} +``` + +### Logging Errors and Exceptions + +```csharp +public async Task ExecuteJob() +{ + try + { + // Job logic + } + catch (Exception ex) + { + Logger.LogError(ex, "Job execution failed."); + } +} +``` + +Logs will automatically be captured, batched, and published to your EventBus message queue. + +--- + +## Exception Handling + +The `EventBusLoggerProvider` gracefully handles exceptions by logging the error locally (console) and temporarily disabling further event-bus logging (by setting `IsEnabled = false`) upon an unrecoverable error such as message publishing failures: + +Example internal logging upon failure: + +``` +Fatal error: Failed to publish the logs to the event bus. +``` + +--- + +## Dependencies + +This logger depends on the following packages: + +- `Microsoft.Extensions.Logging` +- `Olive` +- `Olive.Logging` + +Install via: + +```powershell +Install-Package Microsoft.Extensions.Logging +Install-Package Olive +Install-Package Olive.Logging +``` + +Additionally, ensure connection and network configurations are valid and reliable for your EventBus infrastructure (Amazon AWS SQS, Azure Service Bus, RabbitMQ, etc.). + +--- + +## Important Notes + +- **Batch Logging**: Logs are sent as batches to enhance performance and reduce network overhead. +- **Queue Reliability**: Ensure that your Event Bus queues are durable and able to persist log messages reliably. +- **Sensitive Data Protection**: Always ensure your logs do not contain sensitive or confidential data. Mask or sanitize any potentially sensitive information before logging. +- **Performance Considerations**: Adjust batching options and frequency carefully according to your application's logging volume to optimize performance. \ No newline at end of file From 2bf49c14cad2cbad4660b56cbca21a819f7540ce Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 4 Mar 2025 12:52:35 +0000 Subject: [PATCH 46/68] Olive.Microservices doc --- docs/Distributed/Olive.Microservices.md | 227 ++++++++++++++++++++++++ 1 file changed, 227 insertions(+) create mode 100644 docs/Distributed/Olive.Microservices.md diff --git a/docs/Distributed/Olive.Microservices.md b/docs/Distributed/Olive.Microservices.md new file mode 100644 index 000000000..0322a202b --- /dev/null +++ b/docs/Distributed/Olive.Microservices.md @@ -0,0 +1,227 @@ +# Olive.Microservices + +## Overview + +The **Olive.Microservices** library simplifies inter-service communication within Olive-based microservices architecture. It provides utility classes and extension methods to connect, configure, and invoke operations or APIs across services easily and consistently. + +Using Olive Microservice, configuring microservices URLs, handling authentication tokens, accessing resources, interacting with APIs (via ApiClient), and applying caching and resilience strategies is intuitive. + +--- + +## Table of Contents + +- [Features](#features) +- [Installation](#installation) +- [Configuration](#configuration) +- [Core Classes and Methods](#core-classes-and-methods) + - [Microservice Class](#microservice-class) +- [Working with ApiClient](#working-with-apiclient) + - [API Authentication and Headers](#api-authentication-and-headers) +- [Caching and Resilience Strategies](#caching-and-resilience-strategies) + - [Caching with CachePolicy](#caching-with-cachepolicy) + - [Circuit Breaker](#circuit-breaker-pattern) + - [Retries Logic](#retries-logic) +- [Usage Examples](#usage-examples) +- [Dependencies](#dependencies) +- [Important Notes](#important-notes) + +--- + +## Features + +- Simplifies discovery and referencing of services using URL-based configurations. +- Supports easy creation of `ApiClient` instances with service authentication handling. +- Provides clear configuration management options for resources and S3 buckets. +- Integrated caching strategies to optimize API performance and resilience strategies (circuit breakers, retries) for fault tolerance. + +--- + +## Installation + +Install the Olive libraries using NuGet: + +```powershell +Install-Package Olive +``` + +Ensure you also have required dependencies such as: + +- `Microsoft.Extensions.Hosting` +- `Microsoft.Extensions.DependencyInjection` + +--- + +## Configuration + +Configure microservices URLs and keys in `appsettings.json`: + +```json +{ + "Authentication": { + "Cookie": { "Domain": "https://yourapp.com" } + }, + "Aws": { "Region": "eu-west-1" }, + "Blob": { + "S3": { + "Bucket": "your-bucket", + "Region": "eu-west-1" + } + }, + "Microservice": { + "Posts": { + "Url": "https://posts.yourapp.com/", + "S3BucketUrl": "https://bucket-url/", + "AccessKey": "SERVICE_ACCESS_TOKEN" + }, + "Identity": { + "Url": "https://identity.yourapp.com/" + }, + "Me": { + "Name": "MyService", + "Url": "https://myservice.yourapp.com/" + } + } +} +``` + +--- + +## Core Classes and Methods + +### Microservice Class + +The `Microservice` class encapsulates logic for: + +- Discovering URLs to access other services. +- Creating ApiClient with authentication headers. +- Retrieving resource and asset URLs for services. + +#### Main Properties: +- `Name` - Application service identifier. +- URL access: `Url`, `GetResourceUrl`, `GetS3BucketUrl` + +### Factory Methods: +- `Microservice.Of("service name")` to get another service instance. +- `Microservice.Me` to get currently executing service. + +Example usage: + +```csharp +var identityService = Microservice.Of("Identity"); + +// get full URL for specific path +var loginUrl = identityService.Url("Account/Login"); +``` + +--- + +## Working with ApiClient + +`ApiClient` makes invoking microservice APIs easy and streamlined, handling common concerns such as authentication headers automatically. + +#### API Authentication and Headers + +Example of creating a secured API client: + +```csharp +var apiClient = Microservice.Of("Identity").Api("Account/GetUser"); +``` + +The above setup automatically adds an authentication header (`Microservice.AccessKey`) if configured. + +--- + +## Caching and Resilience Strategies + +Enhance your microservice robustness using caching mechanisms and resilience patterns provided out-of-box. + +### Caching with CachePolicy + +Set caching strategies using `CachePolicy`: + +```csharp +// Cached result up to 5 minutes +proxy.Cache(CachePolicy.FreshOrCache, TimeSpan.FromMinutes(5)); +``` + +### Circuit Breaker Pattern + +Circuit breakers help isolate failing services. If a service repeatedly fails, it temporarily stops making requests: + +```csharp +proxy.CircuitBreaker(exceptionsBeforeBreaking: 3, breakDurationSeconds: 15); +``` + +This prevents cascading failures in microservices architectures. + +### Retries Logic + +Avoid transient errors by automatically retrying failed requests: + +```csharp +proxy.Retries(3, pauseMilliseconds: 500); +``` + +--- + +## Usage Examples + +### Getting a URL from a Microservice Instance: + +```csharp +var postsService = Microservice.Of("Posts"); + +// Get Resources URL (CSS, JS, Media) +string stylePath = postsService.GetResourceUrl("styles/main.css"); + +// Get S3 Bucket URL for assets +string logoImage = postsService.GetS3BucketUrl("images/logo.png"); +``` + +### Using ApiClient for making secure API calls: + +```csharp +// Create an API Client for "Identity" microservice +var client = Microservice.Of("Identity").Api("Users/GetDetails"); + +// add custom header if desired +client.Header(h => h.Add("X-Custom-Header", "value")); + +var user = await client.Get(); +``` + +--- + +## Dependencies + +Make sure you have the required packages installed before use: + +- Olive (`Install-Package Olive`) +- Microsoft Extensions: + - Hosting (`Microsoft.Extensions.Hosting`) + - DependencyInjection (`Microsoft.Extensions.DependencyInjection`) + +```powershell +Install-Package Microsoft.Extensions.Hosting +Install-Package Microsoft.Extensions.DependencyInjection +``` + +--- + +## Exception Handling + +- Robust exception handling for missing configurations or invalid URLs. +- Informative logs and exceptions provided for issues such as: + + ``` + "No queue url is specified in either EventBusLoggerOptions or under config key of Logging:EventBus:QueueUrl" + ``` + +--- + +## Important Notes + +- Ensure configurations (`appsettings.json`) are properly set up for each microservice entry. +- Secure your access keys and connection details carefully. Utilize secret management or environment variables for sensitive data. +- Circuit Breaker and Retry settings should be chosen carefully based on application requirements and external service reliability. +- Ensure assets or resources are correctly configured with appropriate permissions (e.g., AWS S3) to avoid runtime failures. \ No newline at end of file From 1b238d25cf570d1fea9757afb3ecc54502a2fc20 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 4 Mar 2025 14:22:32 +0000 Subject: [PATCH 47/68] Olive.Mvc.CKEditorFileManager doc --- .../Services/Olive.Mvc.CKEditorFileManager.md | 225 ++++++++++++++++++ 1 file changed, 225 insertions(+) create mode 100644 docs/Services/Olive.Mvc.CKEditorFileManager.md diff --git a/docs/Services/Olive.Mvc.CKEditorFileManager.md b/docs/Services/Olive.Mvc.CKEditorFileManager.md new file mode 100644 index 000000000..4883a9ae2 --- /dev/null +++ b/docs/Services/Olive.Mvc.CKEditorFileManager.md @@ -0,0 +1,225 @@ +# Olive.Mvc.CKEditorFileManager + +## Overview + +The Olive CKEditor File Manager component provides robust functionality for managing file uploads, browsing, and downloading within applications that utilize the CKEditor rich-text editor. It simplifies integrating file-management capabilities seamlessly with CKEditor, enabling developers to implement quickly and efficiently. + +--- + +## Table of Contents + +- [Key Features](#key-features) +- [Getting Started](#getting-started) + - [Installation](#installation) + - [Registration and Configuration](#registration-and-configuration) +- [Implementation Guide](#implementation-guide) + - [Defining the File Entity](#defining-the-file-entity) + - [File Uploading](#file-uploading) + - [File Browsing](#file-browsing) + - [File Downloading](#file-downloading) +- [Core Classes](#core-classes) + - [FileManagerController](#filemanagercontroller) + - [Extension Methods](#extension-methods-mvc-registration) + - [ICKEditorFile Interface](#ickeditorfile-interface) + - [ViewModel and DTO](#viewmodel-and-dto-classes) +- [Usage Examples](#usage-examples) +- [Dependencies](#dependencies) + +--- + +## Key Features + +- **Easy CKEditor Integration**: Works smoothly with the popular CKEditor HTML editor for file uploads/downloads within your editor instances. +- **File Upload Management**: Allows secure, efficient file uploads directly from CKEditor. +- **View and Download Files**: Comes with built-in functionalities to browse and securely download stored files. +- **Database Storage Abstraction**: Seamlessly integrates with Olive's entity system for storing and managing files via the DB adapter. + +--- + +## Getting Started + +### Installation + +For your Olive application, ensure the following package is installed to manage CKEditor files: + +``` +Install-Package Olive.Mvc.CKEditorFileManager +``` + +### Registration and Configuration + +In your ASP.NET Core application's `Startup.cs`: + +1. Register the CKEditor File Manager with MVC: + +```csharp +protected override void ConfigureMvc(IMvcBuilder mvc) +{ + ... + mvc.AddCKEditorFileManager(); + ... +} +``` + +2. Define and register the implementer of `ICKEditorFile` interface (an Olive IEntity): + +See the [Defining the File Entity](#defining-the-file-entity) section below. + +--- + +## Implementation Guide + +### Defining the File Entity + +Define your custom entity implementing `ICKEditorFile` to store CKEditor files in your database: + +```csharp +class CKEditorFile : EntityType +{ + public CKEditorFile() + { + Implements("Olive.Mvc.CKEditorFileManager.ICKEditorFile"); + + SecureFile("File"); + } +} +``` + +### File Uploading + +Upload files directly to the database via the built-in controller (`FileManagerController`) provided by this library. + +- Uploads automatically store received files as Blob entities implementing `ICKEditorFile`. + +File upload is mapped to the route: + +``` +/ckeditorfileupload +``` + +Configure CKEditor to use this route for file uploads. + +### File Browsing + +The file browsing feature lets users browse files previously uploaded via CKEditor. + +Access the file browser at: + +``` +/ckeditorfilebrowser +``` + +Displays a simple view of uploaded files for easy selection. Integrate this with your CKEditor instance for image selection, hyperlink insertion, etc. + +### File Downloading + +Files can be securely downloaded using the URL: + +``` +/ckeditorfiledownload/{filename} +``` + +These URLs are safely generated and included automatically within the file browsing view. + +--- + +## Core Classes + +### FileManagerController + +Handles the uploading, browsing, and downloading of CKEditor files: + +- **Upload**: Handles POST requests at `/ckeditorfileupload`. +- **Browser**: Handles GET requests at `/ckeditorfilebrowser`. +- **Download**: Handles GET requests at `/ckeditorfiledownload/{filename}`. + +### Extension Methods (MVC Registration) + +Provides easy registration in your `Startup.cs`: + +```csharp +services.AddMvc().AddCKEditorFileManager(); +``` + +This ensures controllers from Olive.MVC.CKEditorFileManager are properly loaded. + +### ICKEditorFile Interface + +A simple Olive entity interface defining the CKEditor file descriptor: + +```csharp +public interface ICKEditorFile : IEntity +{ + Blob File { get; set; } +} +``` + +### ViewModel and DTO Classes + +These classes (`ViewModel`, `DownloadableFileDto`) hold the file metadata and download URLs when displaying the file browser: + +```csharp +public class ViewModel : IViewModel +{ + public DownloadableFileDto[] Files { get; set; } +} + +public class DownloadableFileDto +{ + public ICKEditorFile CKEditorFile { get; set; } + public string Uri { get; set; } +} +``` + +--- + +## Usage Examples + +### Submitting a File to CKEditor: + +Finally, add the following lines to your CKEditor's config file. + +```javascript +CKEDITOR.editorConfig = function (config) { + ... + config.filebrowserBrowseUrl = '/ckeditorfilebrowser'; + config.filebrowserUploadUrl = '/ckeditorfileupload'; + + // optional: you can set the size for the brower window as following. + config.filebrowserWindowWidth = '300'; + config.filebrowserWindowHeight = '50%'; +} +``` + +### Accessing Uploaded Files: + +Add the `CKEditorFileBrowser.cshtml` view file to your shared view folder contianing somthing like bellow. This file could contain anything you like but `@model Olive.Mvc.CKEditorFileManager.ViewModel`, `class="ckeditor-file-uri"` and `data-download-uri="@item.Uri"` are mandatories part. +```cshtml +@model Olive.Mvc.CKEditorFileManager.ViewModel +@{Layout = "~/Views/Layouts/FrontEnd.Modal.cshtml";} + +
+ +
+``` + +--- + +## Dependencies + +To use Olive CKEditor File Manager, add these dependencies via NuGet or your project file: +- Olive +- Olive.Entities +- Microsoft.Extensions.DependencyInjection +- Microsoft.AspNetCore.Mvc + +``` +Install-Package Olive.Mvc.CKEditorFileManager +Install-Package Olive +Install-Package Olive.Entities +``` \ No newline at end of file From a8d71c1ea6fca9f40bb4ddbd83de89a912651329 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 4 Mar 2025 14:48:19 +0000 Subject: [PATCH 48/68] Olive.Mvc.lpFilter doc --- docs/Services/Olive.Mvc.lpFilter.md | 234 ++++++++++++++++++++++++++++ 1 file changed, 234 insertions(+) create mode 100644 docs/Services/Olive.Mvc.lpFilter.md diff --git a/docs/Services/Olive.Mvc.lpFilter.md b/docs/Services/Olive.Mvc.lpFilter.md new file mode 100644 index 000000000..f01d73247 --- /dev/null +++ b/docs/Services/Olive.Mvc.lpFilter.md @@ -0,0 +1,234 @@ +# Olive.Mvc.lpFilter + +## Overview +The **Olive.Mvc.lpFilter** component provides robust IP-based access control for web applications built on the Olive Framework. It allows you to restrict or permit access to your application based on IP addresses, IP ranges, countries, and geographical regions, implementing strong security measures to control access precisely according to your business requirements. + +--- + +## Table of Contents + +- [Overview](#overview) +- [Key Features](#key-features) +- [Installation](#installation) +- [How IP Filter Works](#how-ip-filter-works) +- [Core Functionalities](#core-functionalities) + - [Global IP Policy](#global-ip-policy) + - [Country-Level Policy](#country-level-policy) + - [Region-Level Policy](#region-level-policy) + - [Specific IP Address Policies](#specific-ip-address-policies) +- [Usage Examples](#usage-examples) +- [Integration with ASP.NET Core](#integration-with-aspnet-core) +- [Country and Region Codes](#country-and-region-codes) +- [Exception Handling](#exception-handling) +- [Dependencies](#dependencies) +- [Security Information](#security-information) +- [Important Notes](#important-notes) + +--- + +## Key Features + +- **Flexible Policies**: Block or allow access based on global settings, specific IP addresses, country codes, or entire regions. +- **Country & Region Blocking**: Utilizes an external DB-IP CSV database to block IP ranges of specific regions or countries. +- **Circuit-breaker-like Resilience**: Ensures IP block lists are loaded only once and cached to reduce overhead. +- **Highly Customizable**: Easily customizable blocked-response pages. +- **Easy Integration with ASP.NET Core Middleware**: Simple hooks to apply IP filtering across your request pipeline. + +--- + +## Installation + +Add dependencies to your Olive project by installing these required packages: + +```powershell +Install-Package Olive +Install-Package Olive.Mvc +``` + +### IP Database Setup + +The IP filter relies on an external CSV database (DB-IP CSV database) to map IP addresses to countries. Download and put the file named **dbip-country.csv** from [DB-IP](https://db-ip.com/db/download/country) into the following directory under your Olive web application root: + +``` +YOUR_PROJECT_ROOT/wwwroot/--IPFilter/dbip-country.csv +``` + +--- + +## How IP Filter Works + +- Incoming Request → Extract client's IP address → Check against global rules, country-based rules, region-based rules, and specific IP rules → Allow or disallow access. +- Uses a powerful IP address storage mechanism (`BigInteger`) ensuring fast and reliable range checks for IP address security. + +--- + +## Core Functionalities + +### Global IP Policy +Set a global allowance or disallowance policy for all incoming connections: + +```csharp +// Allow all IPs globally (default): +IpFilter.SetGlobalPolicy(IpFilter.Policy.Allow); + +// or Disallow all IPs globally: +IpFilter.SetGlobalPolicy(IpFilter.Policy.Disallow); +``` + +### Country-Level Policy +Block or allow IP addresses from specific countries: +```csharp +// Disallow access from specific countries: +IpFilter.SetCountryPolicy(IpFilter.Policy.Disallow, "CN", "RU", "IR"); + +// Allow access for specific countries overriding previous block (countries are identified by 2-letter ISO code): +IpFilter.SetCountryPolicy(IpFilter.Policy.Allow, "GB", "US"); +``` + +### Region-Level Policy +Easily restrict entire global regions with region codes: + +```csharp +// Disallow IPs from entire regions: +IpFilter.SetRegionPolicy(IpFilter.Policy.Disallow, IpFilter.Region.Europe, IpFilter.Region.Asia); + +// or Allow from entire regions: +IpFilter.SetRegionPolicy(IpFilter.Policy.Allow, IpFilter.Region.NorthAmerica); +``` + +### Specific IP Address Policies +Specify IP addresses explicitly, overriding all references from regional or country-based rules: + +```csharp +// Specifically allow certain IPs: +IpFilter.SetSpecificIpPolicy(IpFilter.Policy.Allow, "8.8.8.8"); + +// or specifically disallow certain IPs: +IpFilter.SetSpecificIpPolicy(IpFilter.Policy.Disallow, "1.2.3.4"); +``` + +--- + +## Usage Examples + +### Integration Example in ASP.NET Core pipeline (`Startup.cs`) +Easily integrate IP filtering into your `Configure` method: + +```csharp +app.Use(async (context, next) => +{ + await IpFilter.BlockIfNecessary(context); // Check if current request should be blocked. + await next(); +}); +``` + +### Set a blocking message or redirect page +Customize the response shown to users upon blocked requests as follows: + +```csharp +IpFilter.BlockedAttemptResponse = "Access Restricted: Your region is restricted."; +``` + +### Customize Blocked Actions +Invoke a custom action instead of default message return: + +```csharp +IpFilter.OnBlockedAccessAttempt = async () => +{ + // Log additional information, notify admins, redirect etc. + await Context.Current.Response().Redirect("/blockedregion"); +}; +``` + +--- + +## Integration with ASP.NET Core +You can simply integrate this into your middleware pipeline: + +```csharp +public void Configure(IApplicationBuilder app) +{ + app.Use(async (context, next) => + { + await IpFilter.BlockIfNecessary(context); + await next.Invoke(); + }); +} +``` + +--- + +## Country and Region Codes +The IP Filter uses standard ISO two-letter-code abbreviations for country restriction and predefined region codes: + +| Region | Code | +|--------|------| +| Africa | AF | +| Antarctica | AN | +| Asia | AS | +| Europe | EU | +| North America | NA | +| Oceania | OC | +| South America | SA | + +--- + +## Exception Handling + +Provides clear exceptions on IP-address parsing failures or if required configurations are missing: + +- Missing CSV DB-IP database file will throw exception: + + ``` + Could not find the file 'dbip-country.csv'. + ``` + +- Invalid IP address parsing errors provided clearly: + + ``` + Cannot convert the specified IP address string of 'INVALID-IP' to unit IP address value. + ``` + +--- + +## Dependencies +To use Olive IP Filter, integrate these packages: + +- Olive Framework (`Install-Package Olive`) +- Entities Management (`Install-Package Olive.Entities`) +- Olive.Mvc (`Install-Package Olive.Mvc`) +- CSV handling via Olive.Csv (`Install-Package Olive.Csv`) + +Ensure dependencies are added consecutively in your Olive web application setup, and these Nuget packages are included: + +```powershell +Install-Package Olive +Install-Package Olive.Mvc +Install-Package Olive.Csv +``` + +--- + +## Security Information + +Ensure your IP Filtering settings themselves are protected carefully. Any privileged exceptions (such as office IP addresses) or policy configurations may have security implications if improperly exposed or accidentally misconfigured. + +Always ensure logging of any allowed or blocked attempts is meaningful and monitored: + +```csharp +IpFilter.OnBlockedAccessAttempt = async () => +{ + await Context.Current.Response().EndWith("Access blocked due to policy restrictions."); + + // additional logging here +}; +``` + +--- + +## Important Notes + +- Carefully test all your IP filtering rules in staging before deploying to production environments. Misconfigured rules can lock out valid users. +- Ensure your DB-IP CSV file is frequently updated from reliable IP tagging services (DB-IP is recommended). +- When accessing client IP addresses, ensure you handle trusted proxies correctly with forwarded headers middleware to maintain reliable IP checking. +- Periodically review the IP filtering list, policies, and log entries to adapt to legitimate business and operational changes. \ No newline at end of file From 39d643a21be4f2bd17a384f0d6296e24027e66e7 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 4 Mar 2025 15:36:51 +0000 Subject: [PATCH 49/68] Olive.Mvc.Paging doc --- docs/Services/Olive.Mvc.Paging.md | 251 ++++++++++++++++++++++++++++++ 1 file changed, 251 insertions(+) create mode 100644 docs/Services/Olive.Mvc.Paging.md diff --git a/docs/Services/Olive.Mvc.Paging.md b/docs/Services/Olive.Mvc.Paging.md new file mode 100644 index 000000000..381a33a85 --- /dev/null +++ b/docs/Services/Olive.Mvc.Paging.md @@ -0,0 +1,251 @@ +# Olive MVC Pagination Documentation + +## Overview + +The **Olive MVC Pagination** component introduces robust, easy-to-integrate pagination functionalities tailored for Olive MVC-based applications. It simplifies the presentation and navigation of large datasets by limiting displayed records per page, providing configurable pagination controls, supporting advanced AJAX interactions, and enabling seamless integration with database queries. + +--- + +## Table of Contents +- [Overview](#overview) +- [Features](#features) +- [Installation](#installation) +- [Integration](#integration) +- [Main Classes](#main-classes) + - [`ListPagination`](#listpagination-class) + - [`PaginationRenderer`](#paginationrenderer-class) + - [`ListPaginationBinder`](#listpaginationbinder-class) +- [Core Methods & Properties](#core-methods--properties) +- [Extension Methods](#extension-methods) +- [Usage Examples](#usage-examples) + - [Basic Pagination Usage](#basic-pagination-usage) + - [Advanced Customization](#advanced-customization) + - [Integration with ASP.NET MVC](#integration-with-aspnet-mvc) +- [Dependencies](#dependencies) +- [Exception Handling](#exception-handling) +- [Important Notes](#important-notes) + +--- + +## Features + +- **Easy pagination**: Straightforward implementation for lists, grids, and database queries. +- **Flexible configuration**: Supports customization of page descriptions, layout, navigation controls, and options. +- **AJAX integration**: Can seamlessly integrate with AJAX POST or GET requests for improved user experience. +- **Support MVC Model Binding**: Automatically binds pagination parameters from query strings and forms. +- **Integrated with Olive queries**: Works smoothly with Olive Entity Framework to paginate results directly from the database. +- **Dynamic Page sizes**: Supports selectable predefined page size options. + +--- + +## Installation + +Install via NuGet Package Manager in Visual Studio or CLI: + +```powershell +Install-Package Olive.Mvc +``` + +This component depends on: + +- Olive +- Olive.Entities +- Microsoft.AspNetCore.Mvc +- Microsoft.AspNetCore.Mvc.ViewFeatures + +Ensure these dependencies are also within your project requirements. + +--- + +## Integration + +To integrate pagination with your Olive-based MVC app: + +### Step 1: Add pagination to your ViewModel + +```csharp +public class PostListViewModel : IViewModel +{ + public ListPagination Paging { get; set; } + public IEnumerable Posts { get; set; } +} +``` + +### Step 2: Apply page size and page numbering logic in your controller actions + +```csharp +public async Task List(PostListViewModel model) +{ + var query = Database.Of(); + + model.Paging.TotalItemsCount = await query.Count(); + + model.Posts = await query.Page(model.Paging).GetList(); + + return View(model); +} +``` + +### Step 3: Display pagination controls in your Razor view + +```razor +@using Olive.Mvc; +@model PostListViewModel + +
+ @foreach(var post in Model.Posts) + { +
@post.Title
+ } +
+ +@Html.Pagination(Model.Paging) +``` + +--- + +## Main Classes + +### `ListPagination` Class + +Manages pagination parameters such as `PageSize`, `CurrentPage`, and total count of items. + +Key properties include: + +| Property | Description | +|----------|-------------| +| `CurrentPage` | Current active page number | +| `PageSize` | Number of items per page | +| `TotalItemsCount` | Total data set count | +| `Prefix` | Optional prefix to distinguish pagination parameters | +| `SizeOptions` | List of available page sizes | +| `LastPage` | Computed property for determining the last page number | + +### `PaginationRenderer` Class + +Generates HTML for pagination controls. Integrates with Ajax and traditional navigation. + +### `ListPaginationBinder` Class + +Enables automatic binding of `ListPagination` from incoming query strings or form data in MVC actions. + +--- + +## Core Methods & Properties + +Methods of particular interest include: + +```csharp +void AddPageSizeOptions(params object[] options) +PagingQueryOption ToQueryOption(string orderBy = null) +``` + +**Example Usage**: + +```csharp +model.Paging.AddPageSizeOptions(10, 25, 50, 100); +var queryOption = model.Paging.ToQueryOption("DateCreated DESC"); +``` + +--- + +## Extension Methods + +Include extension methods to easily paginate query results: + +- `TakePage` - For enumerables, applies page-based slicing to a collection. +- `Pagination` - For Razor views, renders pagination interface controls. +- `Page` - For Olive database queries, translates pagination details into Olive query options. + +Example: + +```csharp +var pagedResults = myEnumerable.TakePage(model.Paging); +``` + +or for Olive database queries: + +```csharp +var items = await Database.Of().Page(model.Paging).GetList(); +``` + +--- + +## Usage Examples + +### Basic Pagination Usage + +```csharp +// Controller +model.Paging.TotalItemsCount = await Database.Of().Count(); +model.Posts = await Database.Of().Page(model.Paging).GetList(); +``` + +```razor +// Razor View +@Html.Pagination(Model.Paging) +``` + +### Advanced Customization + +Set custom navigation symbols or icons: + +```csharp +ListPagination.DefaultFirstText = "<<"; +ListPagination.DefaultPreviousText = "<"; +ListPagination.DefaultNextText = ">"; +ListPagination.DefaultLastText = ">>"; +``` + +Enable advanced pagination controls such as First/Last or Previous/Next buttons: + +```csharp +ListPagination.DefaultShowFirstLastLinks = true; +ListPagination.DefaultShowPreviousNextLinks = true; +``` + +### Integration with ASP.NET MVC + +Easily integrate pagination with models via MVC model binding: + +```csharp +public IActionResult Index(MyModel model) +{ + model.Paging.TotalItemsCount = // total items to paginate + model.Items = myService.DoQuery(model.Paging); + + return View(model); +} +``` + +--- + +## Dependencies + +Make sure the following packages are installed and referenced: + +```powershell +Install-Package Olive.Mvc +Install-Package Olive.Entities +Install-Package Microsoft.AspNetCore.Mvc +Install-Package Microsoft.AspNetCore.Mvc.ViewFeatures +``` + +--- + +## Exception Handling + +The pagination library handles typical exceptions gracefully by defaults to fallback values if invalid page numbers or sizes are requested. + +Ensure you consider additional logic to handle exceptional situations or logging appropriately: + +- Invalid or negative page requests default back to page 1. +- Invalid page sizes ignore the faulty size and optionally default to a fallback size or unpaged result. + +--- + +## Important Notes + +- Always ensure the pagination query logic considers sorting and indexing implications for optimal DB efficiency. +- Customize pagination appearance and behaviors through overriding CSS or changing default text. +- Confirm the integration and behavior of AJAX pagination control thoroughly before production usage. \ No newline at end of file From b8639b1b49038a0d32ccc0efaf82adc11e335b9d Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Tue, 4 Mar 2025 16:17:57 +0000 Subject: [PATCH 50/68] Olive.Mvc.Recaptcha doc --- docs/Services/Olive.Mvc.Recaptcha.md | 177 +++++++++++++++++++++++++++ 1 file changed, 177 insertions(+) create mode 100644 docs/Services/Olive.Mvc.Recaptcha.md diff --git a/docs/Services/Olive.Mvc.Recaptcha.md b/docs/Services/Olive.Mvc.Recaptcha.md new file mode 100644 index 000000000..7893b9717 --- /dev/null +++ b/docs/Services/Olive.Mvc.Recaptcha.md @@ -0,0 +1,177 @@ +# Olive.Mvc.Recaptcha + +## Overview + +The Olive reCAPTCHA module adds Google reCAPTCHA services to Olive MVC-based ASP.NET Core web applications. It provides server-side validation of user-submitted reCAPTCHA responses, ensuring robust protection for web forms against bots and spam. The component supports seamless integration of reCAPTCHA in both client-side and server-side contexts, aligns with ASP.NET Core MVC filters, and ensures secure access with clear localization and configuration capabilities. + +--- + +## Table of Contents + +- [Features](#features) +- [Installation](#installation) +- [Configuration](#configuration) +- [Key Components](#key-components) +- [Exception Handling](#exception-handling-and-logging) +- [Dependencies](#dependencies) +- [Important Notes](#important-notes) + +--- + +## Features + +- **Google reCAPTCHA Integration**: Quickly integrate Google reCAPTCHA with your Olive MVC application. +- **Server-side Validation**: Securely handles validation of reCAPTCHA responses against Google's verification endpoint. +- **Localization Support**: Automatically aligns with application locale settings. +- **MVC Filters Integration**: Easily utilize MVC authorization filter pipeline for automatic validation. +- **Configurable and Extensible**: Simple configuration through appsettings.json or programmatic initialization, suitable for extensive customizations. +- **Comprehensive Error Handling and Logging**: Provides structured errors and logging about reCAPTCHA processes for improved monitoring and diagnostics. + +--- + +## Installation + +Use NuGet Package Manager to add the Olive reCAPTCHA integration: + +```powershell +Install-Package Olive.Mvc +``` + +Ensure you also prefer the standard ASP.NET Core dependencies: + +- `Microsoft.AspNetCore.Mvc` +- `Microsoft.Extensions.Logging` +- `Newtonsoft.Json` + +--- + +## Configuration + +Configure your `appsettings.json` with Google’s provided keys: + +```json +{ + "Recaptcha": { + "SiteKey": "[YOUR_SITE_KEY_FROM_GOOGLE]", + "SecretKey": "[YOUR_SECRET_KEY_FROM_GOOGLE]" + } +} +``` + +Or configure programmatically in your `Startup.cs`: + +```csharp +services.AddRecaptcha(options => +{ + options.SiteKey = "[YOUR_SITE_KEY_FROM_GOOGLE]"; + options.SecretKey = "[YOUR_SECRET_KEY_FROM_GOOGLE]"; + options.Enabled = true; +}); +``` + +- Import the tag helpers from **_ViewImport.cshtml** like `@addTagHelper *, Olive.Mvc.Recaptcha`. +- Add the javascript tag in your layout like: +```html +@if (!Request.IsAjaxRequest()) +{ + + +} +``` +- Add captcha to your module with somethig like `CustomField().ControlMarkup("");`. + +--- + +## Key Components + +### RecaptchaService Class + +Manages the validation logic against Google reCAPTCHA API, ensuring secure, robust, and reliable validation for submitted reCAPTCHA tokens. + +Key methods: + +- `Task ValidateResponseAsync(string response, string remoteIp)` - verifies the reCAPTCHA response. +- Provides structured exception handling detailing the verification outcomes. + +### RecaptchaOptions Class + +Holds configuration options necessary for the Olive reCAPTCHA operation. + +| Property | Description | +|----------------------|--------------------------------------------| +| `SiteKey` | Public site key given by Google | +| `SecretKey` | Secret key for validation at your server | +| `Enabled` | Enables or disables reCAPTCHA validation | +| `JavaScriptUrl` | URL for reCAPTCHA script | +| `ValidationMessage` | Custom message displayed when validation fails| +| `LanguageCode` | Sets specific reCAPTCHA locale | + +### ValidateRecaptchaAttribute Attribute + +A MVC action attribute to protect controller actions with reCAPTCHA validation seamlessly. + +```csharp +[ValidateRecaptcha] +public IActionResult SubmitForm(MyViewModel model) +{ + // Your action logic here +} +``` + +Or validate you action with an attribute: +```csharp +Button("Register").IsDefault() + .ExtraActionMethodAttributes("[ValidateRecaptcha]"); +``` + +### RecaptchaValidationException Class + +Encapsulates structured error information for reCAPTCHA validation errors and code: + +```csharp +public class RecaptchaValidationException : Exception +{ + public bool InvalidResponse { get; private set; } +} +``` + +--- + +## Exception Handling and Logging + +The logger captures informative exceptions on validation failures: + +- Provides clear logs identifying what went wrong during Google reCAPTCHA requests. +- Reports errors to assist debugging and configuration corrections when necessary. +- Example logged validation exception: + +``` +Exception: "Looks up a localized string similar to Recaptcha validation failed. The response parameter is missing." +``` + +--- + +## Dependencies + +Include these in your Olive and ASP.NET Core MVC projects: + +- ASP.NET Core MVC (`Microsoft.AspNetCore.Mvc`) +- Microsoft.Extensions.Logging +- Newtonsoft.Json (for JSON serialization) + +Use NuGet to ensure they're properly included: + +```powershell +Install-Package Olive.Mvc +Install-Package Microsoft.Extensions.Logging +Install-Package Newtonsoft.Json +``` + +--- + +## Important Notes + +- Always secure your configuration keys; avoid exposing them publicly. +- Customize reCAPTCHA appearance to match site design through provided options. +- Localization support is direct, specify language via `LanguageCode` or accept automatic client culture assignment (default behavior). +- Consider network latency when adjusting the BackchannelTimeout to avoid unnecessary exception handling due to premature timeouts. \ No newline at end of file From 9f26ec10debbc8577cf1dc365154b748618c57d1 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Wed, 5 Mar 2025 07:25:48 +0000 Subject: [PATCH 51/68] Olive.Mvc.Security doc --- docs/Services/Olive.Mvc.Security.md | 267 ++++++++++++++++++++++++++++ 1 file changed, 267 insertions(+) create mode 100644 docs/Services/Olive.Mvc.Security.md diff --git a/docs/Services/Olive.Mvc.Security.md b/docs/Services/Olive.Mvc.Security.md new file mode 100644 index 000000000..029ded42b --- /dev/null +++ b/docs/Services/Olive.Mvc.Security.md @@ -0,0 +1,267 @@ +# Olive.Mvc.Security + +This document provides an overview and usage examples for the public classes and methods in the `Olive.Security` namespace and related extensions in the Olive framework. It focuses on functionality available to users, such as data protection, authentication, and JWT token management in an ASP.NET Core environment. + +--- + +## Table of Contents + +1. [SymmetricKeyDataProtector](#symmetrickeydataprotector) + - [Overview](#symmetrickeydataprotector-overview) + - [Methods](#symmetrickeydataprotector-methods) +2. [OliveSecurityExtensions](#olivesecurityextensions) + - [Overview](#olivesecurityextensions-overview) + - [Methods](#olivesecurityextensions-methods) +3. [OAuth](#oauth) + - [Overview](#oauth-overview) + - [Methods](#oauth-methods) +4. [JwtAuthenticateAttribute](#jwtauthenticateattribute) + - [Overview](#jwtauthenticateattribute-overview) + - [Methods](#jwtauthenticateattribute-methods) +5. [ILoginInfo](#ilogininfo) + - [Overview](#ilogininfo-overview) +6. [GenericLoginInfo](#genericlogininfo) + - [Overview](#genericlogininfo-overview) +7. [ExternalLoginInfo](#externallogininfo) + - [Overview](#externallogininfo-overview) + +--- + +## SymmetricKeyDataProtector + +### SymmetricKeyDataProtector Overview + +The `SymmetricKeyDataProtector` class implements `IDataProtector` to provide symmetric key-based encryption and decryption with GZip compression. + +### SymmetricKeyDataProtector Methods + +- **`CreateProtector(string purpose)`** + - Creates a new protector instance with a specific purpose. + - **Usage Example:** + ```csharp + var protector = new SymmetricKeyDataProtector("my-secret-key"); + var purposeProtector = protector.CreateProtector("user-data"); + ``` + +- **`Protect(byte[] plaintext)`** + - Encrypts and compresses the provided data. + - **Usage Example:** + ```csharp + var protector = new SymmetricKeyDataProtector("my-secret-key"); + byte[] data = System.Text.Encoding.UTF8.GetBytes("Sensitive info"); + byte[] encrypted = protector.Protect(data); + ``` + +- **`Unprotect(byte[] protectedData)`** + - Decompresses and decrypts the provided data. + - **Usage Example:** + ```csharp + var protector = new SymmetricKeyDataProtector("my-secret-key"); + byte[] encrypted = protector.Protect(System.Text.Encoding.UTF8.GetBytes("Sensitive info")); + byte[] decrypted = protector.Unprotect(encrypted); + Console.WriteLine(System.Text.Encoding.UTF8.GetString(decrypted)); // Outputs: "Sensitive info" + ``` + +--- + +## OliveSecurityExtensions + +### OliveSecurityExtensions Overview + +The `OliveSecurityExtensions` static class offers extension methods for authentication, JWT token creation, and user management. + +### OliveSecurityExtensions Methods + +- **`ToClaimsIdentity(this ILoginInfo @this)`** + - Converts an `ILoginInfo` to a `ClaimsIdentity`. + - **Usage Example:** + ```csharp + ILoginInfo user = new GenericLoginInfo { DisplayName = "John Doe", ID = "123" }; + var identity = user.ToClaimsIdentity(); + ``` + +- **`CreateJwtToken(this ILoginInfo @this, IEnumerable additionalClaims = null, bool remember = false)`** + - Generates a JWT token for the user. + - **Usage Example:** + ```csharp + ILoginInfo user = new GenericLoginInfo { DisplayName = "John Doe", ID = "123" }; + var extraClaims = new List { new Claim("Custom", "Value") }; + string token = user.CreateJwtToken(extraClaims, remember: true); + Console.WriteLine(token); + ``` + +- **`LogOn(this ILoginInfo @this, IEnumerable additionalClaims = null, bool remember = false)`** + - Signs in a user asynchronously. + - **Usage Example:** + ```csharp + ILoginInfo user = new GenericLoginInfo { DisplayName = "John Doe", ID = "123" }; + await user.LogOn(remember: true); + ``` + +- **`IsPersistent(this ClaimsPrincipal @this)`** + - Checks if the user’s session is persistent. + - **Usage Example:** + ```csharp + var principal = new ClaimsPrincipal(new ClaimsIdentity(new[] { new Claim(ClaimTypes.IsPersistent, "true") }, "Olive")); + bool isPersistent = principal.IsPersistent(); // Returns true + ``` + +- **`GetExpiration(this ClaimsPrincipal @this)`** + - Gets the session expiration time. + - **Usage Example:** + ```csharp + var principal = new ClaimsPrincipal(new ClaimsIdentity(new[] { new Claim(ClaimTypes.Expiration, DateTimeOffset.UtcNow.AddDays(1).ToString()) }, "Olive")); + DateTimeOffset expiration = principal.GetExpiration(); + Console.WriteLine(expiration); + ``` + +- **`Is(this ILoginInfo @this, ClaimsPrincipal loggedInUser)`** + - Checks if the `ILoginInfo` ID matches the logged-in user’s ID. + - **Usage Example:** + ```csharp + ILoginInfo user = new GenericLoginInfo { ID = "123" }; + var principal = new ClaimsPrincipal(new ClaimsIdentity(new[] { new Claim(ClaimTypes.NameIdentifier, "123") }, "Olive")); + bool isMatch = user.Is(principal); // Returns true + ``` + +- **`Is(this ClaimsPrincipal @this, ILoginInfo loginInfo)`** + - Checks if the logged-in user’s ID matches the `ILoginInfo` ID. + - **Usage Example:** + ```csharp + var principal = new ClaimsPrincipal(new ClaimsIdentity(new[] { new Claim(ClaimTypes.NameIdentifier, "123") }, "Olive")); + ILoginInfo user = new GenericLoginInfo { ID = "123" }; + bool isMatch = principal.Is(user); // Returns true + ``` + +- **`Clone(this ILoginInfo @this, Action change = null)`** + - Clones an `ILoginInfo` with optional modifications. + - **Usage Example:** + ```csharp + ILoginInfo user = new GenericLoginInfo { DisplayName = "John", ID = "123" }; + var clone = user.Clone(c => c.DisplayName = "Jane"); + Console.WriteLine(clone.DisplayName); // Outputs: "Jane" + ``` + +--- + +## OAuth + +### OAuth Overview + +The `OAuth` class manages external logins, JWT decoding, and session operations. + +### OAuth Methods + +- **`LogOff()`** + - Signs out the current user and clears the session. + - **Usage Example:** + ```csharp + await OAuth.Instance.LogOff(); + ``` + +- **`LoginBy(string provider)`** + - Initiates an external login with a provider. + - **Usage Example:** + ```csharp + await OAuth.Instance.LoginBy("Google"); + ``` + +- **`NotifyExternalLoginAuthenticated(ExternalLoginInfo info)`** + - Notifies event subscribers of an external login. + - **Usage Example:** + ```csharp + OAuth.Instance.ExternalLoginAuthenticated += async (info) => Console.WriteLine(info.Args.Email); + var info = new ExternalLoginInfo { Email = "user@example.com", IsAuthenticated = true }; + await OAuth.Instance.NotifyExternalLoginAuthenticated(info); + ``` + +- **`DecodeJwt(string jwt)`** + - Decodes a JWT token into a `ClaimsPrincipal`. + - **Usage Example:** + ```csharp + string jwt = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."; // Valid JWT token + var principal = OAuth.Instance.DecodeJwt(jwt); + if (principal != null) Console.WriteLine(principal.Identity.Name); + ``` + +--- + +## JwtAuthenticateAttribute + +### JwtAuthenticateAttribute Overview + +The `JwtAuthenticateAttribute` is an action filter for authenticating requests via JWT tokens in the `Authorization` header. + +### JwtAuthenticateAttribute Methods + +- **`OnActionExecuting(ActionExecutingContext context)`** + - Authenticates the request using a JWT token before action execution. + - **Usage Example:** + ```csharp + [JwtAuthenticate] + public IActionResult SecureAction() + { + return Ok("Authenticated!"); + } + // Add "Authorization: Bearer " header to the request + ``` + +--- + +## ILoginInfo + +### ILoginInfo Overview + +The `ILoginInfo` interface defines the structure for user login information. Users typically implement this for custom login logic. + +- **Usage Example:** + ```csharp + public class CustomUser : ILoginInfo + { + public string DisplayName => "Alice"; + public string ID => "456"; + public string Email => "alice@example.com"; + public TimeSpan? Timeout => TimeSpan.FromHours(1); + public IEnumerable GetRoles() => new[] { "User" }; + } + ``` + +--- + +## GenericLoginInfo + +### GenericLoginInfo Overview + +The `GenericLoginInfo` class is a concrete implementation of `ILoginInfo` for general use. + +- **Usage Example:** + ```csharp + var user = new GenericLoginInfo + { + DisplayName = "Bob", + ID = "789", + Email = "bob@example.com", + Roles = new[] { "Admin" }, + Timeout = TimeSpan.FromMinutes(30) + }; + ``` + +--- + +## ExternalLoginInfo + +### ExternalLoginInfo Overview + +The `ExternalLoginInfo` class holds data about an external login attempt. + +- **Usage Example:** + ```csharp + var loginInfo = new ExternalLoginInfo + { + IsAuthenticated = true, + Issuer = "Google", + Email = "user@gmail.com", + NameIdentifier = "google-123", + UserName = "User" + }; + ``` \ No newline at end of file From 1758156bbfe1ea85b1914c2d8a30f6ccfb08205c Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Wed, 5 Mar 2025 08:02:18 +0000 Subject: [PATCH 52/68] Olive.Net4x.Compatibility doc --- docs/Services/Olive.Net4x.Compatibility.md | 134 +++++++++++++++++++++ 1 file changed, 134 insertions(+) create mode 100644 docs/Services/Olive.Net4x.Compatibility.md diff --git a/docs/Services/Olive.Net4x.Compatibility.md b/docs/Services/Olive.Net4x.Compatibility.md new file mode 100644 index 000000000..8f178b89c --- /dev/null +++ b/docs/Services/Olive.Net4x.Compatibility.md @@ -0,0 +1,134 @@ +# Olive.Net4x.Compatibility + +This document provides an overview and usage examples for the public classes and methods in the `Olive` namespace related to compatibility and service provision. It is designed to assist developers in integrating Olive with legacy .NET 4.x applications, providing basic service provider functionality, configuration reading, and HTTP client support. + +--- + +## Table of Contents + +1. [BasicOliveServiceProvider](#basicoliveserviceprovider) + - [Overview](#basicoliveserviceprovider-overview) + - [Methods](#basicoliveserviceprovider-methods) +2. [Compatibility](#compatibility) + - [Overview](#compatibility-overview) + - [Methods](#compatibility-methods) +3. [HttpClientFactory](#httpclientfactory) + - [Overview](#httpclientfactory-overview) + - [Methods](#httpclientfactory-methods) +4. [XmlConfigReader](#xmlconfigreader) + - [Overview](#xmlconfigreader-overview) + - [Methods](#xmlconfigreader-methods) + +--- + +## BasicOliveServiceProvider + +### BasicOliveServiceProvider Overview + +The `BasicOliveServiceProvider` class implements `IServiceProvider` to provide a simple dependency injection mechanism for Olive applications, supporting transient and singleton lifetimes. + +### BasicOliveServiceProvider Methods + +- **`GetService(Type serviceType)`** + - Retrieves a service instance of the specified type based on its lifetime. + - **Usage Example:** + ```csharp + services.AddSingleton(); + + + var loggerFactory = Context.Current.GetService(); + Console.WriteLine(loggerFactory != null); // Outputs: True + ``` + +--- + +## Compatibility + +### Compatibility Overview + +The `Compatibility` class provides static methods to initialize Olive in legacy .NET 4.x applications and manage runtime secrets. + +### Compatibility Methods + +- **`Initialize(Action addServices = null)`** + - Initializes the Olive context for legacy applications, setting up basic services like logging and configuration. + - **Usage Example:** + ```csharp + Compatibility.Initialize(services => + { + services.AddTransient(); + }); + ``` + +- **`LoadSecrets(IDictionary secrets)`** + - Loads production runtime secrets into the configuration manager. + - **Usage Example:** + ```csharp + var secrets = new Dictionary + { + { "ConnectionStrings:MyDb", "Server=myServer;Database=myDb;" }, + { "ApiKey", "xyz123" } + }; + Compatibility.LoadSecrets(secrets); + Console.WriteLine(ConfigurationManager.AppSettings["ApiKey"]); // Outputs: "xyz123" + ``` + +--- + +## HttpClientFactory + +### HttpClientFactory Overview + +The `HttpClientFactory` class implements `IHttpClientFactory` to create `HttpClient` instances for HTTP requests in a legacy environment. + +### HttpClientFactory Methods + +- **`CreateClient(string name)`** + - Creates a new `HttpClient` instance (ignores the name parameter in this implementation). + - **Usage Example:** + ```csharp + var factory = new Compatibility.HttpClientFactory(); + var client = factory.CreateClient(); + var response = await client.GetAsync("https://api.example.com"); + Console.WriteLine(response.StatusCode); + ``` + +--- + +## XmlConfigReader + +### XmlConfigReader Overview + +The `XmlConfigReader` class implements `IConfiguration` to read configuration settings from legacy `web.config` or `app.config` files in XML format. + +### XmlConfigReader Methods + +- **`GetReloadToken()`** + - Returns a token for configuration reload notifications (no-op in this implementation). + - **Usage Example:** + ```csharp + var config = new XmlConfigReader(); + var token = config.GetReloadToken(); + Console.WriteLine(token.HasChanged); // Outputs: False + ``` + +- **`GetChildren()`** + - Not supported; throws an exception if called. + - **Usage Example:** + ```csharp + var config = new XmlConfigReader(); + try + { + config.GetChildren(); // Throws NotSupportedException + } + catch (NotSupportedException) { Console.WriteLine("Not supported"); } + ``` + +- **`GetSection(string key)`** + - Retrieves a configuration section for the specified key. + - **Usage Example:** + ```csharp + var config = new XmlConfigReader(); + var section = config.GetSection("MySection"); + Console.WriteLine(section["SubKey"]); // Accesses "MySection:SubKey" from config + ``` \ No newline at end of file From 194b6cc932e8bec111e88f0b8911a2d32376bce6 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Wed, 5 Mar 2025 08:47:10 +0000 Subject: [PATCH 53/68] Olive.OpenAl.Voice doc --- docs/OpenAI/Olive.OpenAl.Voice.md | 37 +++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100644 docs/OpenAI/Olive.OpenAl.Voice.md diff --git a/docs/OpenAI/Olive.OpenAl.Voice.md b/docs/OpenAI/Olive.OpenAl.Voice.md new file mode 100644 index 000000000..b112c00bf --- /dev/null +++ b/docs/OpenAI/Olive.OpenAl.Voice.md @@ -0,0 +1,37 @@ +# Olive.OpenAl.Voice + +This document provides an overview and usage example for the public class and method in the `Olive.OpenAI.Voice` namespace. It enables interaction with OpenAI's audio transcription capabilities, specifically for converting audio data to text using a speech-to-text model. +Ensure that the `OpenAI:Key` configuration is properly set in your application settings before instantiation. + +--- + +## Table of Contents + +1. [OpenAI](#openai) + - [Overview](#openai-overview) + - [Methods](#openai-methods) + +--- + +## OpenAI + +### OpenAI Overview + +The `OpenAI` class provides a simple interface to transcribe audio data into text using OpenAI's audio transcription service. It leverages the `AudioClient` from the `OpenAI.Audio` namespace and requires an API key and model configuration from the application's settings. + +### OpenAI Methods + +- **`ToText(byte[] audio)`** + - Asynchronously converts an audio byte array to text using OpenAI's speech-to-text model. + - **Usage Example:** + ```csharp + // Ensure OpenAI:Key is set in AppSettings or web.config, e.g., "your-openai-api-key" + var openAI = new OpenAI(); + + // Example: Load an MP3 audio file as a byte array + byte[] audioData = File.ReadAllBytes("sample.mp3"); + + // Transcribe the audio to text + string text = await openAI.ToText(audioData); + Console.WriteLine(text); // Outputs the transcribed text, e.g., "Hello, this is a test." + ``` \ No newline at end of file From 2b6f52fcb6f3da8d0517741f6cb2007ece09d3e2 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Wed, 5 Mar 2025 09:06:14 +0000 Subject: [PATCH 54/68] Olive.OpenAI doc --- docs/OpenAI/Olive.OpenAI.md | 93 +++++++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100644 docs/OpenAI/Olive.OpenAI.md diff --git a/docs/OpenAI/Olive.OpenAI.md b/docs/OpenAI/Olive.OpenAI.md new file mode 100644 index 000000000..a13c164b2 --- /dev/null +++ b/docs/OpenAI/Olive.OpenAI.md @@ -0,0 +1,93 @@ +# Olive.OpenAI + +This document provides an overview and usage examples for the public class and methods in the `Olive.OpenAI` namespace. It enables interaction with OpenAI's chat API, supporting both single-response and streaming-response modes. A configuration section details the required settings in an `appsettings.json` file. + +--- + +## Table of Contents + +1. [OpenAI](#openai) + - [Overview](#openai-overview) + - [Methods](#openai-methods) +2. [Configuration](#configuration) + +--- + +## OpenAI + +### OpenAI Overview + +The `OpenAI` class provides an interface to OpenAI's chat functionality using the `ChatClient` from the `OpenAI.Chat` namespace. It allows sending messages and receiving responses either as a single string or as a streaming sequence, requiring an API key and optionally a model configuration from `appsettings.json`. + +### OpenAI Methods + +- **`GetResponse(string[] messages, string instruction = null)`** + - Asynchronously sends an array of messages to OpenAI and returns a single response as a string. + - **Usage Example:** + ```csharp + var openAI = new OpenAI(); // Assumes OpenAI:Key is set in appsettings.json + string[] messages = { "Hello, how are you?", "Can you help me today?" }; + string response = await openAI.GetResponse(messages, "Respond in a friendly tone"); + Console.WriteLine(response); // Outputs: "Hi there! I'm doing great, thanks for asking. How can I assist you today?" + ``` + +- **`GetResponseStream(string[] messages, string instruction = null)`** + - Asynchronously streams responses from OpenAI for an array of messages, yielding chunks of text as they arrive. + - **Usage Example:** + ```csharp + var openAI = new OpenAI(); // Assumes OpenAI:Key is set in appsettings.json + string[] messages = { "Tell me a story" }; + await foreach (var chunk in openAI.GetResponseStream(messages, "Tell a short story about a cat")) + { + Console.Write(chunk); // Outputs story chunks as they stream, e.g., "Once" "upon" "a time..." + } + ``` + +--- + +## Configuration + +The `OpenAI` class relies on specific configuration settings stored in an `appsettings.json` file with a JSON structure. Below are the required and optional configurations: + +- **`OpenAI:Key`** (Required) + - The API key for authenticating with OpenAI's services. + - **Example:** + ```json + { + "OpenAI": { + "Key": "your-openai-api-key" + } + } + ``` + - If not provided via constructor or config, an `ArgumentException` will be thrown. + +- **`OpenAI:Models:ChatModel`** (Optional) + - Specifies the chat model to use (defaults to `"gpt-4o"` if not set). + - **Example:** + ```json + { + "OpenAI": { + "Key": "your-openai-api-key", + "Models": { + "ChatModel": "gpt-3.5-turbo" + } + } + } + ``` + +### Full `appsettings.json` Example +```json +{ + "OpenAI": { + "Key": "your-openai-api-key", + "Models": { + "ChatModel": "gpt-3.5-turbo" + } + } +} +``` + +### Notes +- If you pass an API key directly to the `OpenAI` constructor (e.g., `new OpenAI("your-key")`), it will override the `OpenAI:Key` from `appsettings.json`. +- Ensure the API key is valid and has appropriate permissions for chat functionality. +- The `Olive.Config.Get` method is assumed to read from `appsettings.json` correctly in your application setup. \ No newline at end of file From c7ce837292bc9f1c82002c5ecebe3b960372ae79 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Wed, 5 Mar 2025 09:47:39 +0000 Subject: [PATCH 55/68] Olive.PDF doc --- docs/Services/Olive.PDF.md | 90 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 docs/Services/Olive.PDF.md diff --git a/docs/Services/Olive.PDF.md b/docs/Services/Olive.PDF.md new file mode 100644 index 000000000..306640a74 --- /dev/null +++ b/docs/Services/Olive.PDF.md @@ -0,0 +1,90 @@ +# Olive.PDF + +This document provides an overview and usage examples for the public class and interface in the `Olive.PDF` namespace. It facilitates the conversion of HTML content to PDF format using a configurable converter service. A configuration section specifies the required settings in an `appsettings.json` file. + +--- + +## Table of Contents + +1. [PdfService](#pdfservice) + - [Overview](#pdfservice-overview) + - [Methods](#pdfservice-methods) +2. [IHtml2PdfConverter](#ihtml2pdfconverter) + - [Overview](#ihtml2pdfconverter-overview) + - [Methods](#ihtml2pdfconverter-methods) +3. [Configuration](#configuration) + +--- + +## PdfService + +### PdfService Overview + +The `PdfService` static class provides a method to create an instance of an HTML-to-PDF converter service, based on a type specified in the configuration. It is designed to abstract the creation of PDF conversion services for use in Olive applications. + +### PdfService Methods + +- **`CreateHtml2PdfConverter()`** + - Creates an instance of an `IHtml2PdfConverter` based on the configured converter type. + - **Usage Example:** + ```csharp + // Assumes Olive.Html2Pdf:ConverterType is set in appsettings.json + var converter = PdfService.CreateHtml2PdfConverter(); + byte[] pdfBytes = converter.GetPdfFromUrlBytes("https://example.com"); + File.WriteAllBytes("output.pdf", pdfBytes); // Saves the PDF to a file + ``` + +--- + +## IHtml2PdfConverter + +### IHtml2PdfConverter Overview + +The `IHtml2PdfConverter` interface defines a contract for converting HTML content from a URL into a PDF byte array. Implementations of this interface are instantiated by `PdfService`. + +### IHtml2PdfConverter Methods + +- **`GetPdfFromUrlBytes(string url)`** + - Converts the HTML content at the specified URL into a PDF byte array. + - **Usage Example:** + ```csharp + // Example assuming an implementation is provided + var converter = PdfService.CreateHtml2PdfConverter(); + byte[] pdfData = converter.GetPdfFromUrlBytes("https://example.com"); + Console.WriteLine($"PDF size: {pdfData.Length} bytes"); + ``` + +--- + +## Configuration + +The `PdfService` class relies on a specific configuration setting stored in an `appsettings.json` file with a JSON structure. Below is the optional configuration: + +- **`Olive.Html2Pdf:ConverterType`** (Optional) + - Specifies the fully qualified type name of the `IHtml2PdfConverter` implementation to use. Defaults to `"Geeks.Html2PDF.Winnovative.Html2PdfConverter, Geeks.Html2PDF.Winnovative"` if not set. + - **Example:** + ```json + { + "Olive": { + "Html2Pdf": { + "ConverterType": "MyCustom.Html2PdfConverter, MyCustomAssembly" + } + } + } + ``` + +### Full `appsettings.json` Example +```json +{ + "Olive": { + "Html2Pdf": { + "ConverterType": "Geeks.Html2PDF.Winnovative.Html2PdfConverter, Geeks.Html2PDF.Winnovative" + } + } +} +``` + +### Notes +- If `Olive.Html2Pdf:ConverterType` is not specified, the default type (`Geeks.Html2PDF.Winnovative.Html2PdfConverter`) is used. +- The specified type must implement `IHtml2PdfConverter` and be accessible in the application's runtime environment. If the type cannot be loaded, an exception will be thrown with details about the failure. +- Ensure the assembly containing the converter type is referenced in your project. \ No newline at end of file From f234f41a0ebe37acdf52fb2dffc345624f95e8f6 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Wed, 5 Mar 2025 10:22:13 +0000 Subject: [PATCH 56/68] Olive.PassiveBackgroundTasks doc --- docs/Services/Olive.PassiveBackgroundTasks.md | 107 ++++++++++++++++++ 1 file changed, 107 insertions(+) create mode 100644 docs/Services/Olive.PassiveBackgroundTasks.md diff --git a/docs/Services/Olive.PassiveBackgroundTasks.md b/docs/Services/Olive.PassiveBackgroundTasks.md new file mode 100644 index 000000000..66fd4d807 --- /dev/null +++ b/docs/Services/Olive.PassiveBackgroundTasks.md @@ -0,0 +1,107 @@ +# Olive.PassiveBackgroundTasks + +This document provides an overview and usage examples for the public classes and methods in the `Olive.PassiveBackgroundTasks` namespace. It enables scheduling and execution of background tasks in an ASP.NET Core application using a cron-based approach. A configuration section details the required settings in an `appsettings.json` file. + +--- + +## Table of Contents + +1. [DistributedBackgroundTasksMiddleware](#distributedbackgroundtasksmiddleware) + - [Overview](#distributedbackgroundtasksmiddleware-overview) + - [Methods](#distributedbackgroundtasksmiddleware-methods) +2. [Extensions](#extensions) + - [Overview](#extensions-overview) + - [Methods](#extensions-methods) +3. [IBackgourndTask](#ibackgourndtask) + - [Overview](#ibackgourndtask-overview) +4. [Configuration](#configuration) + +--- + +## DistributedBackgroundTasksMiddleware + +### DistributedBackgroundTasksMiddleware Overview + +The `DistributedBackgroundTasksMiddleware` class is an ASP.NET Core middleware that triggers the execution of scheduled background tasks when a specific HTTP request is received. It supports forced execution and task-specific runs via query parameters. + +### DistributedBackgroundTasksMiddleware Methods + +- **`Invoke(HttpContext httpContext)`** + - Handles HTTP requests to trigger background task execution and returns a report as an HTML response. + - **Usage Example:** + ```csharp + // Trigger all tasks: GET /olive-trigger-tasks + // Force all tasks: GET /olive-trigger-tasks?force=true + // Run specific task: GET /olive-trigger-tasks?key=MyTask + // The middleware will respond with a list of messages, e.g., "Selected tasks: MyTask", "Finished running MyTask" + ``` + +--- + +## Extensions + +### Extensions Overview + +The `Extensions` static class provides extension methods to integrate scheduled background tasks into an ASP.NET Core application's service collection and middleware pipeline. + +### Extensions Methods + +- **`AddScheduledTasks(this IServiceCollection services)`** + - Registers a background task type and its plan in the dependency injection container. + - **Usage Example:** + ```csharp + services.AddScheduledTasks(); + ``` + +--- + +## IBackgourndTask + +### IBackgourndTask Overview + +The `IBackgourndTask` interface defines the contract for a background task entity, which is persisted and managed by the `Engine`. Users typically implement this interface for custom task definitions. + +- **Usage Example:** +```csharp +using MSharp; + +namespace Domain +{ + class BackgroundTask : EntityType + { + public BackgroundTask() + { + Implements("Olive.PassiveBackgroundTasks.IBackgourndTask"); + String("Name").Mandatory().Unique(); + Guid("Executing instance"); + DateTime("Heartbeat"); + DateTime("Last executed"); + Int("Interval in minutes").Mandatory(); + Int("Timeout in minutes").Mandatory(); + } + } +} +``` + +--- + +## Configuration + +The `Olive.PassiveBackgroundTasks` library relies on a specific configuration setting stored in an `appsettings.json` file with a JSON structure. Below is the required configuration: + +- **`Automated.Tasks:Enabled`** (Required) + - A boolean flag to enable or disable the scheduled tasks feature. If set to `false`, the middleware and task registration will be skipped. + - **Example:** + ```json + { + "Automated": { + "Tasks": { + "Enabled": true + } + } + } + ``` + +### Notes +- If `Automated.Tasks:Enabled` is not set or is `false`, the `UseScheduledTasks` method will bypass task registration and middleware setup. +- The cron expressions used in `BackgroundJobsPlan` (e.g., `"0 * * * *"` for hourly) determine task intervals, parsed by `CronParser`. \ No newline at end of file From 4d58d29d24273d98160134adc65fd89bfa0f5455 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Wed, 5 Mar 2025 10:52:34 +0000 Subject: [PATCH 57/68] Olive.PushNotification doc --- docs/Distributed/Olive.PushNotification.md | 244 +++++++++++++++++++++ 1 file changed, 244 insertions(+) create mode 100644 docs/Distributed/Olive.PushNotification.md diff --git a/docs/Distributed/Olive.PushNotification.md b/docs/Distributed/Olive.PushNotification.md new file mode 100644 index 000000000..96edfcefa --- /dev/null +++ b/docs/Distributed/Olive.PushNotification.md @@ -0,0 +1,244 @@ +# Olive.PushNotification + +This document provides an overview and usage examples for the public classes and methods in the `Olive.PushNotification` namespace. It enables sending push notifications to iOS, Android, and Windows devices using platform-specific brokers. A configuration section details the required settings in an `appsettings.json` file. + +--- + +## Table of Contents + +1. [PushNotificationService](#pushnotificationservice) + - [Overview](#pushnotificationservice-overview) + - [Methods](#pushnotificationservice-methods) +2. [PushNotificationExtensions](#pushnotificationextensions) + - [Overview](#pushnotificationextensions-overview) + - [Methods](#pushnotificationextensions-methods) +3. [IUserDevice](#iuserdevice) + - [Overview](#iuserdevice-overview) +4. [ISubscriptionIdResolver](#isubscriptionidresolver) + - [Overview](#isubscriptionidresolver-overview) + - [Methods](#isubscriptionidresolver-methods) +5. [IPushNotificationService](#ipushnotificationservice) + - [Overview](#ipushnotificationservice-overview) + - [Methods](#ipushnotificationservice-methods) +6. [Configuration](#configuration) + +--- + +## PushNotificationService + +### PushNotificationService Overview + +The `PushNotificationService` class implements `IPushNotificationService` to send push notifications and update badge counts across iOS, Android, and Windows devices using platform-specific brokers from PushSharp. + +### PushNotificationService Methods + +- **`Send(string messageTitle, string messageBody, IEnumerable devices)`** + - Sends a push notification with a title and body to the specified devices. + - **Usage Example:** + ```csharp + var resolver = new NullSubscriptionIdResolver(); + var service = new PushNotificationService(logger, resolver); + var devices = new List + { + new MyUserDevice { DeviceType = "iOS", PushNotificationToken = "ios-token" }, + new MyUserDevice { DeviceType = "Android", PushNotificationToken = "android-token" } + }; + bool success = service.Send("Hello", "This is a test notification", devices); + Console.WriteLine($"Notification sent: {success}"); + ``` + +- **`UpdateBadge(int badge, IEnumerable devices)`** + - Updates the badge count on the specified devices (currently implemented for iOS only). + - **Usage Example:** + ```csharp + + var resolver = new NullSubscriptionIdResolver(); + var service = new PushNotificationService(logger, resolver); + var devices = new List + { + new MyUserDevice { DeviceType = "iOS", PushNotificationToken = "ios-token" } + }; + bool success = service.UpdateBadge(5, devices); + Console.WriteLine($"Badge updated: {success}"); + ``` + +--- + +## PushNotificationExtensions + +### PushNotificationExtensions Overview + +The `PushNotificationExtensions` static class provides extension methods to register push notification services in an ASP.NET Core dependency injection container. + +### PushNotificationExtensions Methods + +- **`AddPushNotification(this IServiceCollection @this)`** + - Registers `PushNotificationService` with a default `NullSubscriptionIdResolver`. + - **Usage Example:** + ```csharp + + services.AddPushNotification(); + + var pushService = Context.Current.GetService(); + ``` + +- **`AddPushNotification(this IServiceCollection @this)`** + - Registers `PushNotificationService` with a custom `ISubscriptionIdResolver` implementation. + - **Usage Example:** + ```csharp + public class CustomResolver : ISubscriptionIdResolver + { + public Task ResolveExpiredSubscription(string oldId, string newId) => Task.CompletedTask; + } + + services.AddPushNotification(); + + var pushService = Context.Current.GetService(); + ``` + +--- + +## IUserDevice + +### IUserDevice Overview + +The `IUserDevice` interface defines the structure for a user device that can receive push notifications, specifying its type and token. + +- **Usage Example:** + ```csharp + public class MyUserDevice : IUserDevice + { + public string DeviceType => "iOS"; + public string PushNotificationToken => "ios-device-token"; + } + ``` + +--- + +## ISubscriptionIdResolver + +### ISubscriptionIdResolver Overview + +The `ISubscriptionIdResolver` interface defines a contract for resolving expired subscription IDs when push notifications fail due to token expiration. + +### ISubscriptionIdResolver Methods + +- **`ResolveExpiredSubscription(string oldSubscriptionId, string newSubscriptionId)`** + - Resolves an expired subscription by updating or removing the old ID. + - **Usage Example:** + ```csharp + public class CustomResolver : ISubscriptionIdResolver + { + public async Task ResolveExpiredSubscription(string oldId, string newId) + { + Console.WriteLine($"Replacing {oldId} with {newId ?? "null"}"); + await Task.CompletedTask; + } + } + ``` + +--- + +## IPushNotificationService + +### IPushNotificationService Overview + +The `IPushNotificationService` interface defines the contract for sending push notifications and updating badge counts across supported platforms. + +### IPushNotificationService Methods + +- **`Send(string messageTitle, string messageBody, IEnumerable devices)`** + - Sends a push notification to the specified devices. + - **Usage Example:** See `PushNotificationService.Send` above. + +- **`UpdateBadge(int badge, IEnumerable devices)`** + - Updates the badge count on the specified devices. + - **Usage Example:** See `PushNotificationService.UpdateBadge` above. + +--- + +## Configuration + +The `PushNotificationService` class requires specific configuration settings stored in an `appsettings.json` file with a JSON structure to initialize platform-specific brokers. Below are the settings for each platform: + +### Apple (iOS) +- **`PushNotification:Apple:CertificateFile`** (Required) + - Path to the `.p12` or `.pfx` certificate file for APNs. +- **`PushNotification:Apple:CertificatePassword`** (Required) + - Password for the certificate file. +- **`PushNotification:Apple:Environment`** (Required) + - APNs environment (`Production` or `Sandbox`). +- **Example:** + ```json + { + "PushNotification": { + "Apple": { + "CertificateFile": "Certificates/apns-cert.p12", + "CertificatePassword": "your-password", + "Environment": "Production" + } + } + } + ``` + +### Google (Android) +- **`PushNotification:Google:SenderId`** (Required) + - Sender ID from Google Cloud Messaging (FCM). +- **`PushNotification:Google:AuthToken`** (Required) + - Server key (authentication token) from FCM. +- **Example:** + ```json + { + "PushNotification": { + "Google": { + "SenderId": "your-sender-id", + "AuthToken": "your-auth-token" + } + } + } + ``` + +### Windows +- **`PushNotification:Windows:PackageName`** (Required) + - Package name from the Windows Dev Center. +- **`PushNotification:Windows:PackageSID`** (Required) + - Security Identifier (SID) for the package. +- **`PushNotification:Windows:ClientSecret`** (Required) + - Client secret from the Windows Dev Center. +- **Example:** + ```json + { + "PushNotification": { + "Windows": { + "PackageName": "your-package-name", + "PackageSID": "your-package-sid", + "ClientSecret": "your-client-secret" + } + } + } + ``` + +### Full `appsettings.json` Example +```json +{ + "PushNotification": { + "Apple": { + "CertificateFile": "Certificates/apns-cert.p12", + "CertificatePassword": "your-password", + "Environment": "Production" + }, + "Google": { + "SenderId": "your-sender-id", + "AuthToken": "your-auth-token" + }, + "Windows": { + "PackageName": "your-package-name", + "PackageSID": "your-package-sid", + "ClientSecret": "your-client-secret" + } + } +} +``` + +### Notes +- Each platform's broker is initialized only if its respective configuration is provided. If a configuration is missing (e.g., `CertificateFile` for Apple), that platform will be skipped. From b936a1cca2637af6c1a64bedc94a5d0e083a70ff Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Wed, 5 Mar 2025 11:45:24 +0000 Subject: [PATCH 58/68] Olive.RabbitMQ doc --- docs/RabbitMQ/Olive.RabbitMQ.md | 188 ++++++++++++++++++++++++++++++++ 1 file changed, 188 insertions(+) create mode 100644 docs/RabbitMQ/Olive.RabbitMQ.md diff --git a/docs/RabbitMQ/Olive.RabbitMQ.md b/docs/RabbitMQ/Olive.RabbitMQ.md new file mode 100644 index 000000000..9f487a865 --- /dev/null +++ b/docs/RabbitMQ/Olive.RabbitMQ.md @@ -0,0 +1,188 @@ +# Olive RabbitMQ Event Bus Library Documentation + +This document provides an overview and usage examples for the public classes and methods in the `Olive.RabbitMQ` namespace. It provides an event bus implementation using RabbitMQ for publishing and consuming messages, supporting both single and batch operations. A configuration section details the required settings in an `appsettings.json` file. + +--- + +## Table of Contents + +1. [EventBusProvider](#eventbusprovider) + - [Overview](#eventbusprovider-overview) + - [Methods](#eventbusprovider-methods) +2. [EventBusQueue](#eventbusqueue) + - [Overview](#eventbusqueue-overview) + - [Methods](#eventbusqueue-methods) +3. [RabbitMQEventBusExtensions](#rabbitmqeventbusextensions) + - [Overview](#rabbitmqeventbusextensions-overview) + - [Methods](#rabbitmqeventbusextensions-methods) +4. [Configuration](#configuration) + +--- + +## EventBusProvider + +### EventBusProvider Overview + +The `EventBusProvider` class implements `IEventBusQueueProvider` to provide instances of `EventBusQueue`, caching them by queue URL for efficient reuse. + +### EventBusProvider Methods + +- **`Provide(string queueUrl)`** + - Returns an `IEventBusQueue` instance for the specified queue URL, creating a new one if not cached. + - **Usage Example:** + ```csharp + var provider = new EventBusProvider(); + var queue = provider.Provide("my-queue"); + await queue.Publish("Hello, RabbitMQ!"); + ``` + +--- + +## EventBusQueue + +### EventBusQueue Overview + +The `EventBusQueue` class implements `IEventBusQueue` to interact with a RabbitMQ queue, supporting message publishing, batch publishing, subscription, and pulling messages. + +### EventBusQueue Methods + +- **`Publish(string message)`** + - Publishes a single message to the queue. + - **Usage Example:** + ```csharp + var queue = new EventBusQueue("my-queue"); + await queue.Publish("Single message"); + ``` + +- **`PublishBatch(IEnumerable messages)`** + - Publishes a batch of messages to the queue. + - **Usage Example:** + ```csharp + var queue = new EventBusQueue("my-queue"); + var messages = new[] { "Message 1", "Message 2", "Message 3" }; + await queue.PublishBatch(messages); + ``` + +- **`Subscribe(Func handler)`** + - Subscribes to the queue with a handler function that processes incoming messages. + - **Usage Example:** + ```csharp + var queue = new EventBusQueue("my-queue"); + queue.Subscribe(async message => + { + Console.WriteLine($"Received: {message}"); + await Task.CompletedTask; + }); + ``` + +- **`PullAll(Func handler)`** + - Pulls all messages from the queue and processes them with the provided handler until the queue is empty. + - **Usage Example:** + ```csharp + var queue = new EventBusQueue("my-queue"); + await queue.PullAll(async message => + { + Console.WriteLine($"Pulled: {message}"); + await Task.CompletedTask; + }); + ``` + +- **`PullBatch(int timeoutSeconds = 10, int? maxNumerOfMessages = null)`** + - Pulls a batch of messages from the queue with a specified timeout and optional maximum number. + - **Usage Example:** + ```csharp + var queue = new EventBusQueue("my-queue"); + var messages = await queue.PullBatch(timeoutSeconds: 5, maxNumerOfMessages: 5); + foreach (var msg in messages) + { + Console.WriteLine($"Batch message: {msg.RawMessage}"); + await msg.Complete(); + } + ``` + +- **`Pull(int timeoutSeconds = 10)`** + - Pulls a single message from the queue with a specified timeout. + - **Usage Example:** + ```csharp + var queue = new EventBusQueue("my-queue"); + var message = await queue.Pull(timeoutSeconds: 5); + if (message != null) + { + Console.WriteLine($"Single pull: {message.RawMessage}"); + await message.Complete(); + } + ``` + +- **`Purge()`** + - Purges all messages from the queue. + - **Usage Example:** + ```csharp + var queue = new EventBusQueue("my-queue"); + await queue.Purge(); + Console.WriteLine("Queue purged"); + ``` + +--- + +## RabbitMQEventBusExtensions + +### RabbitMQEventBusExtensions Overview + +The `RabbitMQEventBusExtensions` static class provides an extension method to register the RabbitMQ event bus provider in an ASP.NET Core dependency injection container. + +### RabbitMQEventBusExtensions Methods + +- **`AddRabbitMQEventBus(this IServiceCollection @this)`** + - Registers `EventBusProvider` as the `IEventBusQueueProvider` in the service collection. + - **Usage Example:** + ```csharp + + services.AddRabbitMQEventBus(); + + var eventBusProvider = Context.Current.GetService(); + ``` + +--- + +## Configuration + +The `EventBusQueue` class relies on specific configuration settings stored in an `appsettings.json` file with a JSON structure. Below are the required and optional settings: + +### Required Settings +- **`RabbitMQ:Host`** + - The hostname of the RabbitMQ server. +- **`RabbitMQ:Username`** + - The username for RabbitMQ authentication. +- **`RabbitMQ:Password`** + - The password for RabbitMQ authentication. + +### Optional Settings +- **`RabbitMQ:Port`** (Default: `5672`) + - The port number for the RabbitMQ connection. +- **`RabbitMQ:EnableSSL`** (Default: `false`) + - Whether to enable SSL for the RabbitMQ connection. +- **`RabbitMQ:EventBusQueue:MaxNumberOfMessages`** (Default: `10`) + - The maximum number of messages to pull in a batch (1-10). +- **`RabbitMQ:EventBusQueue:VisibilityTimeout`** (Default: `300`) + - The duration (in seconds) that received messages are hidden from subsequent retrievals. + +### Full `appsettings.json` Example +```json +{ + "RabbitMQ": { + "Host": "localhost", + "Username": "guest", + "Password": "guest", + "Port": 5672, + "EnableSSL": false, + "EventBusQueue": { + "MaxNumberOfMessages": 5, + "VisibilityTimeout": 600 + } + } +} +``` + +### Notes +- If any required settings (`Host`, `Username`, `Password`) are missing, the `EventBusQueue` instantiation will fail unless overridden programmatically. +- When `EnableSSL` is `true`, the connection uses TLS 1.1 or 1.2, and server certificate validation is bypassed for simplicity (not recommended for production without proper certificate handling). From 62cc3ad470d5dd9ca051e5a28f1e640b6c1e485f Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Wed, 5 Mar 2025 12:17:41 +0000 Subject: [PATCH 59/68] Olive.SMS.TextMagic doc --- docs/Services/Olive.SMS.TextMagic.md | 79 ++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+) create mode 100644 docs/Services/Olive.SMS.TextMagic.md diff --git a/docs/Services/Olive.SMS.TextMagic.md b/docs/Services/Olive.SMS.TextMagic.md new file mode 100644 index 000000000..57fd4d350 --- /dev/null +++ b/docs/Services/Olive.SMS.TextMagic.md @@ -0,0 +1,79 @@ +# Olive.SMS.TextMagic + +This document provides an overview and usage examples for the public classes and methods in the `Olive.SMS.TextMagic` namespace. It enables sending SMS messages using the TextMagic REST API. A configuration section details the required settings in an `appsettings.json` file. + +--- + +## Table of Contents + +1. [SmsDispatcher](#smsdispatcher) + - [Overview](#smsdispatcher-overview) + - [Methods](#smsdispatcher-methods) +2. [IServiceCollectionExtension](#iservicecollectionextension) + - [Overview](#iservicecollectionextension-overview) + - [Methods](#iservicecollectionextension-methods) +3. [Configuration](#configuration) + +--- + +## SmsDispatcher + +### SmsDispatcher Overview + +The `SmsDispatcher` class implements `Olive.SMS.ISmsDispatcher` to send SMS messages via the TextMagic REST API, utilizing the `TextmagicRest` library for communication. + +### SmsDispatcher Methods + +- **`Dispatch(ISmsMessage sms)`** + - Asynchronously sends an SMS message using the TextMagic API. + - **Usage Example:** + ```csharp + var dispatcher = new SmsDispatcher(); + var sms = new SmsMessage { To = "+1234567890", Text = "Hello from TextMagic!" }; // Assuming SmsMessage implements ISmsMessage + await dispatcher.Dispatch(sms); + Console.WriteLine("SMS sent successfully"); + ``` + +--- + +## IServiceCollectionExtension + +### IServiceCollectionExtension Overview + +The `IServiceCollectionExtension` static class provides an extension method to register the TextMagic SMS dispatcher in an ASP.NET Core dependency injection container. + +### IServiceCollectionExtension Methods + +- **`AddTextMagic(this IServiceCollection @this)`** + - Registers `SmsDispatcher` as the `ISmsDispatcher` in the service collection. + - **Usage Example:** + ```csharp + services.AddTextMagic(); + ``` + +--- + +## Configuration + +The `SmsDispatcher` class requires specific configuration settings stored in an `appsettings.json` file with a JSON structure. Below are the required settings: + +### Required Settings +- **`Sms:TextMagic:Username`** + - The username for authenticating with the TextMagic API. +- **`Sms:TextMagic:Key`** + - The API key (token) for authenticating with the TextMagic API. + +### Full `appsettings.json` Example +```json +{ + "Sms": { + "TextMagic": { + "Username": "your-username", + "Key": "your-api-key" + } + } +} +``` + +### Notes +- Both `Sms:TextMagic:Username` and `Sms:TextMagic:Key` must be provided in the configuration; otherwise, a `Config.GetOrThrow` exception will be thrown during instantiation. \ No newline at end of file From b68c6dcd866e7cb2dc5bd6389ff13e480a432fa7 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Thu, 6 Mar 2025 08:43:16 +0000 Subject: [PATCH 60/68] Olive.Security.Auth0 doc --- docs/Services/Olive.Security.Auth0.md | 80 +++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) create mode 100644 docs/Services/Olive.Security.Auth0.md diff --git a/docs/Services/Olive.Security.Auth0.md b/docs/Services/Olive.Security.Auth0.md new file mode 100644 index 000000000..7f840dc74 --- /dev/null +++ b/docs/Services/Olive.Security.Auth0.md @@ -0,0 +1,80 @@ +# Olive.Security.Auth0 + +This document provides an overview and usage examples for the public class and methods in the `Olive.Security` namespace related to Auth0 authentication. It enables username/password authentication via the Auth0 Authentication API. A configuration section details the required settings in an `appsettings.json` file. + +--- + +## Table of Contents + +1. [Auth0](#auth0) + - [Overview](#auth0-overview) + - [Methods](#auth0-methods) + - [Nested Classes](#auth0-nested-classes) +2. [Configuration](#configuration) + +--- + +## Auth0 + +### Auth0 Overview + +The `Auth0` class provides a static interface to authenticate users against an Auth0 tenant using the `Auth0.AuthenticationApi`. It leverages the Auth0 Authentication API client to perform username/password login requests and returns the result as an `AuthenticationResult`. + +### Auth0 Methods + +- **`Authenticate(string username, string password)`** + - Asynchronously authenticates a user with the provided username and password against Auth0. + - **Usage Example:** + ```csharp + var result = await Auth0.Authenticate("user@example.com", "securepassword"); + if (result.Success) + { + Console.WriteLine("Authentication successful"); + } + else + { + Console.WriteLine($"Authentication failed: {result.Message}"); + } + ``` + +### Auth0 Nested Classes + +- **`AuthenticationResult`** + - Represents the outcome of an authentication attempt. + - **Usage Example:** + ```csharp + var result = await Auth0.Authenticate("user@example.com", "wrongpassword"); + Console.WriteLine($"Success: {result.Success}, Message: {result.Message}"); + // Outputs: Success: False, Message: + ``` + +--- + +## Configuration + +The `Auth0` class requires specific configuration settings stored in an `appsettings.json` file with a JSON structure. Below are the required settings: + +### Required Settings +- **`Authentication:Auth0:ClientId`** + - The Client ID of your Auth0 application, obtained from the Auth0 dashboard. +- **`Authentication:Auth0:ClientSecret`** + - The Client Secret of your Auth0 application, obtained from the Auth0 dashboard. +- **`Authentication:Auth0:Domain`** + - The domain of your Auth0 tenant (e.g., `your-tenant.auth0.com`). + +### Full `appsettings.json` Example +```json +{ + "Authentication": { + "Auth0": { + "ClientId": "your-client-id", + "ClientSecret": "your-client-secret", + "Domain": "your-tenant.auth0.com" + } + } +} +``` + +### Notes +- All three settings (`ClientId`, `ClientSecret`, `Domain`) are mandatory. If any are missing, the `Config.Get` method will throw an exception unless default values are handled elsewhere in your application. +- The Auth0 credentials must correspond to an application configured with the "Username-Password-Authentication" connection enabled in your Auth0 tenant. \ No newline at end of file From 7d0f51a420e754655c7c319f4447b27d5bd665a8 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Thu, 6 Mar 2025 08:52:17 +0000 Subject: [PATCH 61/68] Olive.Security.Aws doc --- docs/Security/Olive.Security.Aws.md | 90 +++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 docs/Security/Olive.Security.Aws.md diff --git a/docs/Security/Olive.Security.Aws.md b/docs/Security/Olive.Security.Aws.md new file mode 100644 index 000000000..6fd515cf4 --- /dev/null +++ b/docs/Security/Olive.Security.Aws.md @@ -0,0 +1,90 @@ +# Olive.Security.Aws + +This document provides an overview and usage examples for the public classes and methods in the `Olive.Security.Aws` namespace. It integrates AWS Key Management Service (KMS) for generating and managing data encryption keys, as well as providing data protection services. A configuration section details the required settings in an `appsettings.json` file or environment variables. + +--- + +## Table of Contents + +1. [DataKeyService](#datakeyservice) + - [Overview](#datakeyservice-overview) + - [Methods](#datakeyservice-methods) +2. [KmsDataProtectionProvider](#kmsdataprotectionprovider) + - [Overview](#kmsdataprotectionprovider-overview) + - [Methods](#kmsdataprotectionprovider-methods) +3. [Configuration](#configuration) + +--- + +## DataKeyService + +### DataKeyService Overview + +The `DataKeyService` class implements `IDataKeyService` to generate and retrieve encryption keys using AWS KMS. It caches decrypted keys for efficient reuse and interacts with KMS to generate and decrypt data keys. + +### DataKeyService Methods + +- **`GenerateKey()`** + - Asynchronously generates a new AES-256 data key from AWS KMS. + - **Usage Example:** + ```csharp + var keyService = new DataKeyService { MasterKeyArn = "arn:aws:kms:region:account-id:key/key-id" }; + var key = await keyService.GenerateKey(); + Console.WriteLine($"Generated key length: {key.EncryptionKey.Length} bytes"); + ``` + +- **`GetEncryptionKey(byte[] encryptionKeyReference)`** + - Retrieves or decrypts an encryption key from a given reference (ciphertext blob) using AWS KMS, caching the result. + - **Usage Example:** + ```csharp + var keyService = new DataKeyService { MasterKeyArn = "arn:aws:kms:region:account-id:key/key-id" }; + var key = await keyService.GenerateKey(); + var encryptionKey = keyService.GetEncryptionKey(key.EncryptionKeyReference); + Console.WriteLine($"Retrieved key length: {encryptionKey.Length} bytes"); + ``` + +--- + +## KmsDataProtectionProvider + +### KmsDataProtectionProvider Overview + +The `KmsDataProtectionProvider` class extends `Cloud.DataProtectionProvider` to provide data protection services using AWS KMS keys. It integrates with ASP.NET Core's data protection stack and configures the master key ARN from configuration or environment variables. + +### KmsDataProtectionProvider Methods + +- **`CreateProtector(string purpose)`** + - Creates a new data protector instance for the specified purpose, leveraging AWS KMS for key management. + - **Usage Example:** + ```csharp + var provider = new KmsDataProtectionProvider(); + var protector = provider.CreateProtector("MyPurpose"); + var protectedData = protector.Protect(System.Text.Encoding.UTF8.GetBytes("Sensitive data")); + Console.WriteLine($"Protected data: {Convert.ToBase64String(protectedData)}"); + ``` + +--- + +## Configuration + +The `DataKeyService` and `KmsDataProtectionProvider` classes require a specific configuration setting for the AWS KMS master key ARN, which can be stored in an `appsettings.json` file or as an environment variable. Below are the details: + +### Required Setting +- **`Aws:Kms:MasterKeyArn`** (or Environment Variable `AWS_KMS_MASTERKEY_ARN`) + - The Amazon Resource Name (ARN) of the AWS KMS master key used for generating and decrypting data keys. + - **Priority:** The environment variable `AWS_KMS_MASTERKEY_ARN` takes precedence over the `appsettings.json` value if both are present. + +### Full `appsettings.json` Example +```json +{ + "Aws": { + "Kms": { + "MasterKeyArn": "arn:aws:kms:us-east-1:123456789012:key/abcd1234-abcd-1234-abcd-1234567890ab" + } + } +} +``` + +### Notes +- The master key ARN must be specified either in `appsettings.json` under `Aws:Kms:MasterKeyArn` or as the `AWS_KMS_MASTERKEY_ARN` environment variable. If neither is provided, an exception will be thrown with the message "Aws Master Key Arn is not specified." +- Ensure the AWS credentials used by the application have permissions to call `GenerateDataKey` and `Decrypt` on the specified KMS key. From 74dad6cfd2eca816478aad010c6f81b3f7d8c698 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Thu, 6 Mar 2025 09:06:03 +0000 Subject: [PATCH 62/68] Olive.Security.Azure doc --- docs/Security/Olive.Security.Azure.md | 124 ++++++++++++++++++++++++++ 1 file changed, 124 insertions(+) create mode 100644 docs/Security/Olive.Security.Azure.md diff --git a/docs/Security/Olive.Security.Azure.md b/docs/Security/Olive.Security.Azure.md new file mode 100644 index 000000000..ef92ea88b --- /dev/null +++ b/docs/Security/Olive.Security.Azure.md @@ -0,0 +1,124 @@ +# Olive.Security.Azure + +This document provides an overview and usage examples for the public classes and methods in the `Olive.Security.Azure` namespace. It integrates Azure Key Vault for key management and data protection, including key generation and encryption, as well as persisting data protection keys to Azure Blob Storage. A configuration section details the required settings in an `appsettings.json` file or environment variables. + +--- + +## Table of Contents + +1. [DataKeyService](#datakeyservice) + - [Overview](#datakeyservice-overview) + - [Methods](#datakeyservice-methods) +2. [Extensions](#extensions) + - [Overview](#extensions-overview) + - [Methods](#extensions-methods) +3. [KeyVaultDataProtectionProvider](#keyvaultdataprotectionprovider) + - [Overview](#keyvaultdataprotectionprovider-overview) + - [Methods](#keyvaultdataprotectionprovider-methods) +4. [Configuration](#configuration) + +--- + +## DataKeyService + +### DataKeyService Overview + +The `DataKeyService` class implements `IDataKeyService` to generate and retrieve encryption keys using Azure Key Vault. It caches decrypted keys for efficient reuse and uses the Azure Key Vault KeyClient and CryptographyClient for key operations. + +### DataKeyService Methods + +- **`GenerateKey()`** + - Asynchronously generates a new encryption key and wraps it using Azure Key Vault with the RSA1_5 algorithm. + - **Usage Example:** + ```csharp + var keyService = new DataKeyService(); + var key = await keyService.GenerateKey(); + Console.WriteLine($"Generated key length: {key.EncryptionKey.Length} bytes"); + ``` + +- **`GetEncryptionKey(byte[] encryptionKeyReference)`** + - Retrieves or unwraps an encryption key from a given reference (encrypted key blob) using Azure Key Vault, caching the result. + - **Usage Example:** + ```csharp + var keyService = new DataKeyService(); + var key = await keyService.GenerateKey(); + var encryptionKey = keyService.GetEncryptionKey(key.EncryptionKeyReference); + Console.WriteLine($"Retrieved key length: {encryptionKey.Length} bytes"); + ``` + +--- + +## Extensions + +### Extensions Overview + +The `Extensions` static class provides an extension method to configure ASP.NET Core data protection to persist keys to Azure Blob Storage. + +### Extensions Methods + +- **`PersistKeysToAzureBlobStorage(this IDataProtectionBuilder builder)`** + - Configures the data protection system to persist keys to Azure Blob Storage using a default Azure credential. + - **Usage Example:** + ```csharp + IDataProtectionBuilder builder; + builder.PersistKeysToAzureBlobStorage(); + ``` + +--- + +## KeyVaultDataProtectionProvider + +### KeyVaultDataProtectionProvider Overview + +The `KeyVaultDataProtectionProvider` class extends `Cloud.DataProtectionProvider` to provide data protection services using Azure Key Vault keys. It integrates with ASP.NET Core's data protection stack. + +### KeyVaultDataProtectionProvider Methods + +- **`CreateProtector(string purpose)`** + - Creates a new data protector instance for the specified purpose, leveraging Azure Key Vault for key management. + - **Usage Example:** + ```csharp + var provider = new KeyVaultDataProtectionProvider(); + var protector = provider.CreateProtector("MyPurpose"); + var protectedData = protector.Protect(System.Text.Encoding.UTF8.GetBytes("Sensitive data")); + Console.WriteLine($"Protected data: {Convert.ToBase64String(protectedData)}"); + ``` + +--- + +## Configuration + +The `DataKeyService` and related classes require specific configuration settings, which can be stored in an `appsettings.json` file or as environment variables. Below are the required settings: + +### Required Settings +- **`Azure:KeyVault:CookieAuthentication:Uri`** (or Environment Variable `AZURE_KEY_VALUT_COOKIE_AUTHENTICATION_URI`) + - The URI of the Azure Key Vault (e.g., `https://your-vault-name.vault.azure.net/`). + - **Priority:** The environment variable takes precedence over the `appsettings.json` value if both are present. +- **`Azure:KeyVault:CookieAuthentication:KeyName`** (or Environment Variable `AZURE_KEY_VALUT_COOKIE_AUTHENTICATION_KEY_NAME`) + - The name of the master key in Azure Key Vault used for wrapping and unwrapping encryption keys. + - **Priority:** The environment variable takes precedence over the `appsettings.json` value if both are present. +- **`Azure:KeyVault:CookieAuthentication:BlobStorageUri`** (or Environment Variable `AZURE_KEY_VALUT_COOKIE_AUTHENTICATION_BLOB_STORAGE_URI`) + - The URI of the Azure Blob Storage container for persisting data protection keys (e.g., `https://your-storage-account.blob.core.windows.net/your-container/`). + - **Priority:** The environment variable takes precedence over the `appsettings.json` value if both are present. + +### Full `appsettings.json` Example +```json +{ + "Azure": { + "KeyVault": { + "CookieAuthentication": { + "Uri": "https://your-vault-name.vault.azure.net/", + "KeyName": "DataProtectionKey", + "BlobStorageUri": "https://your-storage-account.blob.core.windows.net/your-container/" + } + } + } +} +``` + + + +### Notes +- All three settings (`Uri`, `KeyName`, `BlobStorageUri`) are mandatory. If any are missing, an exception will be thrown (e.g., "Azure Key Valut Authentication uri is not specified"). +- The application must have appropriate Azure credentials configured (e.g., via `DefaultAzureCredential`), which may use environment variables (`AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`, `AZURE_TENANT_ID`), managed identity, or other authentication methods supported by Azure.Identity. +- The Azure Key Vault key must support wrapping/unwrapping operations (e.g., RSA key type), and the Blob Storage container must be accessible with write permissions. \ No newline at end of file From ec9d29684a9d5efdfc90eb33c9c79c8309973b0c Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Thu, 6 Mar 2025 09:23:46 +0000 Subject: [PATCH 63/68] Olive.Security.Cloud.CustomEncryption doc --- .../Olive.Security.Cloud.CustomEncryption.md | 79 +++++++++++++++++++ 1 file changed, 79 insertions(+) create mode 100644 docs/Security/Olive.Security.Cloud.CustomEncryption.md diff --git a/docs/Security/Olive.Security.Cloud.CustomEncryption.md b/docs/Security/Olive.Security.Cloud.CustomEncryption.md new file mode 100644 index 000000000..ac56c0ad3 --- /dev/null +++ b/docs/Security/Olive.Security.Cloud.CustomEncryption.md @@ -0,0 +1,79 @@ +# Olive.Security.Cloud.CustomEncryption + +This document provides an overview and usage examples for the public classes and methods in the `Olive.Security.Cloud.CustomEncryption` namespace. It implements a custom encryption-based key management and data protection service using AES encryption, integrated with ASP.NET Core's data protection system. A configuration section details the required environment variable setting. + +--- + +## Table of Contents + +1. [DataKeyService](#datakeyservice) + - [Overview](#datakeyservice-overview) + - [Methods](#datakeyservice-methods) +2. [CustomEncryptionDataProtectionProvider](#customencryptiondataprotectionprovider) + - [Overview](#customencryptiondataprotectionprovider-overview) + - [Methods](#customencryptiondataprotectionprovider-methods) +3. [Configuration](#configuration) + +--- + +## DataKeyService + +### DataKeyService Overview + +The `DataKeyService` class implements `IDataKeyService` to generate and retrieve encryption keys using AES encryption with a master key sourced from an environment variable. It uses a fixed IV derived from the master key and caches keys for efficiency. + +### DataKeyService Methods + +- **`GenerateKey()`** + - Asynchronously generates a new encryption key (a GUID without hyphens) and encrypts it using AES with the master key. + - **Usage Example:** + ```csharp + var keyService = new DataKeyService(); + var key = await keyService.GenerateKey(); + Console.WriteLine($"Generated key length: {key.EncryptionKey.Length} bytes"); + ``` + +- **`GetEncryptionKey(byte[] encryptionKeyReference)`** + - Decrypts an encryption key from its reference (encrypted blob) using AES with the master key. + - **Usage Example:** + ```csharp + var keyService = new DataKeyService(); + var key = await keyService.GenerateKey(); + var encryptionKey = keyService.GetEncryptionKey(key.EncryptionKeyReference); + Console.WriteLine($"Retrieved key length: {encryptionKey.Length} bytes"); + ``` + +--- + +## CustomEncryptionDataProtectionProvider + +### CustomEncryptionDataProtectionProvider Overview + +The `CustomEncryptionDataProtectionProvider` class extends `Cloud.DataProtectionProvider` to provide data protection services using the custom AES-based encryption implemented in `DataKeyService`. It integrates with ASP.NET Core's data protection stack. + +### CustomEncryptionDataProtectionProvider Methods + +- **`CreateProtector(string purpose)`** + - Creates a new data protector instance for the specified purpose, leveraging the custom encryption service. + - **Usage Example:** + ```csharp + var provider = new CustomEncryptionDataProtectionProvider(); + var protector = provider.CreateProtector("MyPurpose"); + var protectedData = protector.Protect(System.Text.Encoding.UTF8.GetBytes("Sensitive data")); + Console.WriteLine($"Protected data: {Convert.ToBase64String(protectedData)}"); + ``` + +--- + +## Configuration + +The `DataKeyService` class requires a specific configuration setting, which is sourced from an environment variable rather than an `appsettings.json` file in this implementation. Below is the required setting: + +### Required Setting +- **`Encryption:MasterKey`** (Environment Variable) + - The master key used for AES encryption and decryption. It must be a valid string that can be converted to bytes using UTF-8 encoding. + - **Note:** The key should ideally be at least 32 characters (for a 256-bit key) to match AES-256 requirements, though the code will use whatever length is provided and derive a 16-byte IV from it. + +### Notes +- The `Encryption:MasterKey` environment variable is mandatory. If it is not set or empty, an exception will be thrown with the message "Could not find the master key." +- The AES implementation uses the master key directly as the encryption key and derives a 16-byte initialization vector (IV) from the first 16 bytes of the key. For security, ensure the master key is sufficiently random and long (e.g., 32 bytes for AES-256). \ No newline at end of file From 2d32be7d2e792fe9489865a463fe654ed136e251 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Thu, 6 Mar 2025 09:32:43 +0000 Subject: [PATCH 64/68] Olive.Security.Cloud doc --- docs/Security/Olive.Security.Cloud.md | 111 ++++++++++++++++++++++++++ 1 file changed, 111 insertions(+) create mode 100644 docs/Security/Olive.Security.Cloud.md diff --git a/docs/Security/Olive.Security.Cloud.md b/docs/Security/Olive.Security.Cloud.md new file mode 100644 index 000000000..6e7239568 --- /dev/null +++ b/docs/Security/Olive.Security.Cloud.md @@ -0,0 +1,111 @@ +# Olive.Security.Cloud + +This document provides an overview and usage examples for the public classes and methods in the `Olive.Security.Cloud` namespace. It defines a foundational framework for cloud-based data protection and key management, abstracting key generation and encryption/decryption operations. A configuration section notes that specific settings depend on the concrete implementation of `IDataKeyService`. + +--- + +## Table of Contents + +1. [DataProtectionProvider](#dataprotectionprovider) + - [Overview](#dataprotectionprovider-overview) + - [Methods](#dataprotectionprovider-methods) +2. [IDataKeyService](#idatakeyservice) + - [Overview](#idatakeyservice-overview) + - [Methods](#idatakeyservice-methods) +3. [Key](#key) + - [Overview](#key-overview) +4. [Configuration](#configuration) + +--- + +## DataProtectionProvider + +### DataProtectionProvider Overview + +The `DataProtectionProvider` abstract class implements `IDataProtector` to provide data protection services using a specified `IDataKeyService` implementation. It generates encryption keys, protects data with a symmetric key protector, and caches decrypted data for efficiency, incorporating GZip compression for protected data. + +### DataProtectionProvider Methods + +- **`CreateProtector(string purpose)`** + - Abstract method to create a new data protector instance for the specified purpose. Must be implemented by derived classes. + +- **`Protect(byte[] plaintext)`** + - Protects (encrypts) the provided plaintext data using a generated key and GZip compression. + - **Usage Example:** + ```csharp + var provider = new MyProvider(); // Assuming MyProvider and MyDataKeyService are implemented + var plaintext = System.Text.Encoding.UTF8.GetBytes("Sensitive data"); + var protectedData = provider.Protect(plaintext); + Console.WriteLine($"Protected data: {Convert.ToBase64String(protectedData)}"); + ``` + +- **`Unprotect(byte[] protectedData)`** + - Unprotects (decrypts) the provided protected data, decompressing it from GZip and caching the result. + - **Usage Example:** + ```csharp + var provider = new MyProvider(); // Assuming MyProvider and MyDataKeyService are implemented + var plaintext = System.Text.Encoding.UTF8.GetBytes("Sensitive data"); + var protectedData = provider.Protect(plaintext); + var unprotectedData = provider.Unprotect(protectedData); + Console.WriteLine($"Unprotected data: {System.Text.Encoding.UTF8.GetString(unprotectedData)}"); + ``` + +--- + +## IDataKeyService + +### IDataKeyService Overview + +The `IDataKeyService` interface defines a contract for key management services, providing methods to generate and retrieve encryption keys. Concrete implementations (e.g., AWS KMS, Azure Key Vault) must provide the actual logic. + +### IDataKeyService Methods + +- **`GenerateKey()`** + - Asynchronously generates a new encryption key and its reference. + - **Usage Example:** (Assuming a concrete implementation) + ```csharp + public class MyDataKeyService : IDataKeyService + { + public async Task GenerateKey() => new Key { EncryptionKey = new byte[16], EncryptionKeyReference = new byte[16] }; + public byte[] GetEncryptionKey(byte[] reference) => new byte[16]; + } + + var service = new MyDataKeyService(); + var key = await service.GenerateKey(); + ``` + +- **`GetEncryptionKey(byte[] encryptionKeyReference)`** + - Retrieves the encryption key corresponding to the provided reference. + - **Usage Example:** (Assuming a concrete implementation) + ```csharp + var service = new MyDataKeyService(); + var key = await service.GenerateKey(); + var encryptionKey = service.GetEncryptionKey(key.EncryptionKeyReference); + ``` + +--- + +## Key + +### Key Overview + +The `Key` class represents a key pair consisting of an encryption key and its encrypted reference, used by `IDataKeyService` implementations to manage encryption operations. + +- **Usage Example:** + ```csharp + var key = new Key + { + EncryptionKey = System.Text.Encoding.UTF8.GetBytes("my-encryption-key"), + EncryptionKeyReference = System.Text.Encoding.UTF8.GetBytes("encrypted-ref") + }; + Console.WriteLine($"Key length: {key.EncryptionKey.Length}"); + ``` + +--- + +## Configuration + +The `DataProtectionProvider` class itself does not directly require configuration settings, as it relies on the specific `TDataKeyService` implementation for key management. Therefore, configuration requirements vary depending on the concrete `IDataKeyService` implementation used (e.g., AWS KMS, Azure Key Vault, custom encryption). + + ### Notes +- To use this library, you must instantiate a concrete `DataProtectionProvider` with an appropriate `IDataKeyService` implementation and ensure its configuration is provided. \ No newline at end of file From 0102b7c17384af882704d8a4de52340573e066a6 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Thu, 6 Mar 2025 10:04:03 +0000 Subject: [PATCH 65/68] Olive.Security.Impersonation doc --- docs/Security/Olive.Security.Impersonation.md | 99 +++++++++++++++++++ 1 file changed, 99 insertions(+) create mode 100644 docs/Security/Olive.Security.Impersonation.md diff --git a/docs/Security/Olive.Security.Impersonation.md b/docs/Security/Olive.Security.Impersonation.md new file mode 100644 index 000000000..6b6e9ae3a --- /dev/null +++ b/docs/Security/Olive.Security.Impersonation.md @@ -0,0 +1,99 @@ +# Olive.Security.Impersonation + +This document provides an overview and usage examples for the public classes and methods in the `Olive.Security` namespace related to user impersonation. It enables an admin user to impersonate another user, manage the impersonation session, and retrieve impersonation-related information in an ASP.NET Core application. + +--- + +## Table of Contents + +1. [ImpersonationSession](#impersonationsession) + - [Overview](#impersonationsession-overview) + - [Methods](#impersonationsession-methods) +2. [ImpersonationExtensions](#impersonationextensions) + - [Overview](#impersonationextensions-overview) + - [Methods](#impersonationextensions-methods) +3. [Configuration](#configuration) + +--- + +## ImpersonationSession + +### ImpersonationSession Overview + +The `ImpersonationSession` static class provides methods to manage user impersonation in an ASP.NET Core application. It allows an admin user to impersonate another user, end the impersonation, and display a widget indicating the impersonation status. + +### ImpersonationSession Methods + +- **`IsImpersonated()`** + - Determines if the current user is in an impersonation session by checking for a specific role. + - **Usage Example:** + ```csharp + bool isImpersonated = await ImpersonationSession.IsImpersonated(); + Console.WriteLine($"User is impersonated: {isImpersonated}"); + ``` + +- **`Impersonate(ILoginInfo user)`** + - Initiates an impersonation session for the specified user, preserving the original user's claims as impersonator claims. + - **Usage Example:** + ```csharp + var userToImpersonate = new GenericLoginInfo + { + DisplayName = "John Doe", + ID = "123", + Email = "john.doe@example.com", + Roles = new[] { "User" } + }; + await ImpersonationSession.Impersonate(userToImpersonate); + Console.WriteLine("Impersonation started"); + ``` + +- **`EndImpersonation()`** + - Ends the current impersonation session and restores the original user's identity. + - **Usage Example:** + ```csharp + if (await ImpersonationSession.IsImpersonated()) + { + await ImpersonationSession.EndImpersonation(); + Console.WriteLine("Impersonation ended"); + } + ``` + +- **`GetWidget()`** + - Returns an HTML snippet indicating the impersonation status if the user is impersonated. + - **Usage Example:** + ```csharp + var widget = await ImpersonationSession.GetWidget(); + Console.WriteLine(widget); // Outputs: "
Impersonating John Doe
" or empty string + ``` + +--- + +## ImpersonationExtensions + +### ImpersonationExtensions Overview + +The `ImpersonationExtensions` static class provides extension methods for working with `ClaimsPrincipal` objects during impersonation, including converting claims to and from impersonator format and retrieving the impersonator's name. + +### ImpersonationExtensions Methods + +- **`GetImpersonatorName(this ClaimsPrincipal @this)`** + - Retrieves the name of the original user (impersonator) from the claims if the current user is impersonated. + - **Usage Example:** + ```csharp + var user = Context.Current.Http().User; // Assuming an impersonated user + var impersonatorName = user.GetImpersonatorName(); + Console.WriteLine($"Impersonator: {impersonatorName}"); + ``` + +--- + +## Configuration + +The `ImpersonationSession` and `ImpersonationExtensions` classes do not require specific configuration settings from an `appsettings.json` file or environment variables beyond the standard Olive framework setup. They rely on the following prerequisites: + +### Prerequisites + - **Olive Framework Setup:** + - The authentication system must be configured to support `SignInAsync` and `SignOutAsync` methods (typically via ASP.NET Core authentication middleware). + +### Notes + - The impersonation role (`Olive-IMPERSONATOR`) is hardcoded and automatically added to the impersonated user's roles. \ No newline at end of file From 9d7dd9cde5df3a5ed53cd6df18d0c9af7f097d0b Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Thu, 6 Mar 2025 10:48:01 +0000 Subject: [PATCH 66/68] Olive.Sms.Aws doc --- docs/AWS/Olive.Sms.Aws.md | 80 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) create mode 100644 docs/AWS/Olive.Sms.Aws.md diff --git a/docs/AWS/Olive.Sms.Aws.md b/docs/AWS/Olive.Sms.Aws.md new file mode 100644 index 000000000..2049d304e --- /dev/null +++ b/docs/AWS/Olive.Sms.Aws.md @@ -0,0 +1,80 @@ +# Olive.Sms.Aws + +This document provides an overview and usage examples for the public classes and methods in the `Olive.SMS` namespace related to sending SMS messages via Amazon Simple Notification Service (SNS). It integrates with AWS SNS to dispatch SMS messages, supporting optional sender ID and origination number configurations. A configuration section details the optional settings in an `appsettings.json` file. + +--- + +## Table of Contents + +1. [SmsDispatcher](#smsdispatcher) + - [Overview](#smsdispatcher-overview) + - [Methods](#smsdispatcher-methods) +2. [IServiceCollectionExtension](#iservicecollectionextension) + - [Overview](#iservicecollectionextension-overview) + - [Methods](#iservicecollectionextension-methods) +3. [Configuration](#configuration) + +--- + +## SmsDispatcher + +### SmsDispatcher Overview + +The `SmsDispatcher` class implements `ISmsDispatcher` and `IDisposable` to send SMS messages using Amazon SNS. It uses the `AmazonSimpleNotificationServiceClient` to publish messages and supports optional attributes like sender ID and origination number. + +### SmsDispatcher Methods + +- **`Dispatch(ISmsMessage sms)`** + - Asynchronously sends an SMS message via AWS SNS with configurable attributes. + - **Usage Example:** + ```csharp + var dispatcher = Context.Current.GetService(); + var sms = new SmsMessage { To = "+1234567890", Text = "Hello from AWS SNS!" }; // Assuming SmsMessage implements ISmsMessage + await dispatcher.Dispatch(sms); + Console.WriteLine("SMS sent successfully"); + ``` + +--- + +## IServiceCollectionExtension + +### IServiceCollectionExtension Overview + +The `IServiceCollectionExtension` static class provides an extension method to register the AWS SNS SMS dispatcher in an ASP.NET Core dependency injection container. + +### IServiceCollectionExtension Methods + +- **`AddAwsSms(this IServiceCollection @this)`** + - Registers `SmsDispatcher` as the `ISmsDispatcher` in the service collection. + - **Usage Example:** + ```csharp + services.AddAwsSms(); + ``` + +--- + +## Configuration + +The `SmsDispatcher` class supports optional configuration settings stored in an `appsettings.json` file with a JSON structure. These settings enhance the SMS sending process but are not strictly required. Below are the optional settings: + +### Optional Settings +- **`Aws:Sns:SenderId`** + - The sender ID to display on the recipient's device (e.g., a brand name). Overrides `ISmsMessage.SenderName` if specified in the message. +- **`Aws:Sns:OriginationNumber`** + - The phone number to use as the origination number for the SMS (must be an AWS SNS-supported number). + +### Full `appsettings.json` Example +```json +{ + "Aws": { + "Sns": { + "SenderId": "MyApp", + "OriginationNumber": "+12025550123" + } + } +} +``` + +### Notes +- If `SenderId` or `OriginationNumber` are not provided in `appsettings.json`, the dispatcher will use the `SenderName` from the `ISmsMessage` (if available) or omit these attributes. +- The SMS type is hardcoded to `"Transactional"` for immediate delivery, suitable for critical notifications. From 5e010a618e5e569f881accefdd1e2c8247daf69d Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Thu, 6 Mar 2025 11:01:17 +0000 Subject: [PATCH 67/68] Olive.Web doc --- docs/Core/Olive.Web.md | 269 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 269 insertions(+) create mode 100644 docs/Core/Olive.Web.md diff --git a/docs/Core/Olive.Web.md b/docs/Core/Olive.Web.md new file mode 100644 index 000000000..bf76ec3e5 --- /dev/null +++ b/docs/Core/Olive.Web.md @@ -0,0 +1,269 @@ +# Olive.Web + +This document provides an overview and usage examples for the public classes and methods in the `Olive`, `Olive.Web`, and `Olive.Mvc` namespaces. These extensions enhance web-related functionality in ASP.NET Core applications, including API authentication, HTTP request/response handling, cookie management, caching, and ETag support. + +--- + +## Table of Contents + +1. [OliveWebExtensions](#olivewebextensions) + - [Overview](#olivewebextensions-overview) + - [Methods](#olivewebextensions-methods) +2. [CookieProperty](#cookieproperty) + - [Overview](#cookieproperty-overview) + - [Methods](#cookieproperty-methods) +3. [HttpContextCache](#httpcontextcache) + - [Overview](#httpcontextcache-overview) + - [Methods](#httpcontextcache-methods) +4. [SupportETagCacheAttribute](#supportetagcacheattribute) + - [Overview](#supportetagcacheattribute-overview) + - [Methods](#supportetagcacheattribute-methods) + +--- + +## OliveWebExtensions + +### OliveWebExtensions Overview + +The `OliveWebExtensions` static class provides a collection of extension methods for enhancing web functionality, including API client authentication, HTTP context access, request/response utilities, and more, within the Olive framework. + +### OliveWebExtensions Methods + +- **`AsHttpUser(this ApiClient @this)`** + - Authenticates an `ApiClient` instance with the current user's cookies. + - **Usage Example:** + ```csharp + var client = new ApiClient("https://api.example.com"); + client.AsHttpUser(); + var response = await client.Get("endpoint"); + ``` + +- **`User(this Context context)`** + - Retrieves the current user's `ClaimsPrincipal`. + - **Usage Example:** + ```csharp + var user = Context.Current.User(); + Console.WriteLine(user?.Identity?.Name); + ``` + +- **`Http(this Context context)`** + - Retrieves the current `HttpContext`. + - **Usage Example:** + ```csharp + var httpContext = Context.Current.Http(); + ``` + +- **`Request(this Context context)`** + - Retrieves the current `HttpRequest`. + - **Usage Example:** + ```csharp + var request = Context.Current.Request(); + ``` + +- **`GetCookies(this HttpRequest @this)`** + - Retrieves all cookies from the request. + - **Usage Example:** + ```csharp + var cookies = Context.Current.Request().GetCookies(); + foreach (var cookie in cookies) Console.WriteLine($"{cookie.Key}: {cookie.Value}"); + ``` + +- **`GetOrDefault(this HttpRequest @this, string key)`** + - Retrieves an entity or value from the request by key, returning default if not found. + - **Usage Example:** + ```csharp + var user = await Context.Current.Request().GetOrDefault("userId"); + ``` + +- **`Get(this HttpRequest @this, string key)`** + - Retrieves an entity or value from the request by key, throwing an exception if not found. + - **Usage Example:** + ```csharp + var user = await Context.Current.Request().Get("userId"); + ``` + +- **`GetList(this HttpRequest @this, string key)`** + - Retrieves a list of entities from the request by key, split by comma. + - **Usage Example:** + ```csharp + var users = Context.Current.Request().GetList("userIds"); + foreach (var userTask in users) Console.WriteLine((await userTask)?.ToString()); + ``` + +- **`FindGoogleSearchKeyword(this HttpRequest @this)`** + - Extracts Google search keywords from the referrer URL. + - **Usage Example:** + ```csharp + var keyword = Context.Current.Request().FindGoogleSearchKeyword(); + Console.WriteLine(keyword); // e.g., "olive framework" + ``` + +- **`GetIPAddress(this HttpRequest @this)`** + - Gets the client's IP address. + - **Usage Example:** + ```csharp + var ip = Context.Current.Request().GetIPAddress(); + Console.WriteLine(ip); + ``` + +- **`Has(this HttpRequest @this, string argument)`** + - Checks if a parameter exists in the request. + - **Usage Example:** + ```csharp + bool hasParam = Context.Current.Request().Has("key"); + ``` + +- **`RootUrl(this HttpRequest @this)`** + - Gets the root URL of the request. + - **Usage Example:** + ```csharp + var root = Context.Current.Request().RootUrl(); + Console.WriteLine(root); // e.g., "https://example.com/" + ``` + +- **`ToAbsoluteUri(this HttpRequest @this)`** + - Gets the absolute URI of the request. + - **Usage Example:** + ```csharp + var uri = Context.Current.Request().ToAbsoluteUri(); + Console.WriteLine(uri); + ``` + +- **`IsAjaxRequest(this HttpRequest @this)`** + - Determines if the request is an AJAX call. + - **Usage Example:** + ```csharp + bool isAjax = Context.Current.Request().IsAjaxRequest(); + ``` + +- **`RedirectPermanent(this HttpResponse response, string permanentUrl)`** + - Issues a 301 redirect to a permanent URL. + - **Usage Example:** + ```csharp + Context.Current.Response().RedirectPermanent("https://new-url.com"); + ``` + +- **`Dispatch(this HttpResponse response, FileInfo responseFile, string fileName = null, string contentType = "Application/octet-stream")`** + - Sends a file to the client. + - **Usage Example:** + ```csharp + var file = new FileInfo("path/to/file.pdf"); + await Context.Current.Response().Dispatch(file, "download.pdf"); + ``` + +- **`Dispatch(this HttpResponse response, byte[] responseData, string fileName, string contentType = "Application/octet-stream")`** + - Sends binary data as a file to the client. + - **Usage Example:** + ```csharp + var data = System.Text.Encoding.UTF8.GetBytes("Hello"); + await Context.Current.Response().Dispatch(data, "hello.txt"); + ``` + +- **`EndWith(this HttpResponse response, string message, string mimeType = "text/html")`** + - Writes a message and ends the response. + - **Usage Example:** + ```csharp + await Context.Current.Response().EndWith("Done!"); + ``` + +- **`Write(this HttpResponse response, string message)`** + - Writes a message to the response synchronously. + - **Usage Example:** + ```csharp + Context.Current.Response().Write("Hello"); + ``` + +--- + +## CookieProperty + +### CookieProperty Overview + +The `CookieProperty` static class provides methods to manage HTTP cookies, including getting and setting values in the request and response. + +### CookieProperty Methods + +- **`Get(string propertyName = null, T defaultValue = default)`** + - Retrieves a cookie value by key, with an optional default. + - **Usage Example:** + ```csharp + var value = await CookieProperty.Get("MyCookie", "default"); + Console.WriteLine(value); + ``` + +- **`Set(string propertyName = null, T value, bool isHttpOnly = true)`** + - Sets a cookie value. + - **Usage Example:** + ```csharp + CookieProperty.Set("MyCookie", "value"); + ``` + +- **`SetList(string propertyName, IEnumerable list, bool isHttpOnly = true)`** + - Sets a list of entity IDs as a cookie. + - **Usage Example:** + ```csharp + var users = new[] { new User { ID = "1" }, new User { ID = "2" } }; + CookieProperty.SetList("UserIds", users); + ``` + +- **`GetList(string propertyName = null)`** + - Retrieves a list of entities from a cookie. + - **Usage Example:** + ```csharp + var users = await CookieProperty.GetList("UserIds"); + foreach (var user in users) Console.WriteLine(user?.ID); + ``` + +- **`Remove(string propertyName)`** + - Removes a cookie by key. + - **Usage Example:** + ```csharp + CookieProperty.Remove("MyCookie"); + ``` + +--- + +## HttpContextCache + +### HttpContextCache Overview + +The `HttpContextCache` static class provides methods for caching objects within the scope of an HTTP request. + +### HttpContextCache Methods + +- **`GetOrAdd(TKey key, Func valueProducer)`** + - Gets or adds a value to the HTTP context cache. + - **Usage Example:** + ```csharp + var cachedValue = HttpContextCache.GetOrAdd("key", () => "computed value"); + Console.WriteLine(cachedValue); + ``` + +- **`Remove(TKey key)`** + - Removes a value from the HTTP context cache. + - **Usage Example:** + ```csharp + HttpContextCache.Remove("key"); + ``` + +--- + +## SupportETagCacheAttribute + +### SupportETagCacheAttribute Overview + +The `SupportETagCacheAttribute` class is an action filter that enables ETag-based caching for GET requests, returning a `304 Not Modified` response when the client’s ETag matches the server’s. + +### SupportETagCacheAttribute Methods + +- **`OnActionExecuted(ActionExecutedContext context)`** + - Executes after the action, implementing ETag caching logic. + - **Usage Example:** + ```csharp + [SupportETagCache] + public IActionResult GetData() + { + return Ok("Data"); + } + // Client request with matching ETag header returns 304 + ``` \ No newline at end of file From 5d52ea42cb752619fe58b3465258dde9c155d639 Mon Sep 17 00:00:00 2001 From: mansour_nazifiasl_geeksltd Date: Thu, 6 Mar 2025 13:42:12 +0000 Subject: [PATCH 68/68] Olive.Gpt doc --- docs/Services/Olive.Gpt.md | 277 +++++++++++++++++++++++++++++++++++++ 1 file changed, 277 insertions(+) create mode 100644 docs/Services/Olive.Gpt.md diff --git a/docs/Services/Olive.Gpt.md b/docs/Services/Olive.Gpt.md new file mode 100644 index 000000000..4999d39c5 --- /dev/null +++ b/docs/Services/Olive.Gpt.md @@ -0,0 +1,277 @@ +# Olive.Gpt + +This document provides an overview and usage examples for the key public classes and methods in the `Olive.Gpt` namespace. It integrates with OpenAI's API for chat completions, image generation (DALL-E), and assistant management, with optional AWS S3 storage for images. A configuration section details optional settings for S3 integration in an `appsettings.json` file. + +--- + +## Table of Contents + +[Getting Started](#getting-started) +[Api](#api) + - [Overview](#api-overview) + - [Key Methods](#api-key-methods) +[AssistantApi](#assistantapi) + - [Overview](#assistantapi-overview) + - [Key Methods](#assistantapi-key-methods) +[Command](#command) + - [Overview](#command-overview) + - [Key Methods](#command-key-methods) +[Examples](#examples) + +--- +## Getting Started +- You will need an API Key from OpenAI. To obtain yours, you will need to create an account directly on www.openai.com and use that in the application settings file. +- In your project, add this nuget reference: `Olive.Gpt` +- Create an instance of the API: `var gpt = new Olive.Gpt.Api(Config.Get("OpenAIApi"));` +- Invoke `GetResponse(...)` on the api object. + +For Example: +```csharp +var gpt = new Olive.Gpt.Api(Config.Get("OpenAIApi")); +string response = await gpt.GetResponse("Who invented the computer mouse?"); +``` +--- + +## Api + +### Api Overview + +The `Api` class is the primary interface for interacting with OpenAI's chat and image generation APIs. It supports single responses, streaming responses, transformation workflows, and DALL-E image generation with optional S3 storage. + +### Api Key Methods + +- **`GetResponse(string command, string model = null, ResponseFormats responseFormat = ResponseFormats.NotSet, Action streamHandler = null)`** + - Retrieves a single response from OpenAI for a command. + - **Usage Example:** + ```csharp + var api = new Api("your-api-key"); + var response = await api.GetResponse("Write a short story"); + Console.WriteLine(response); + ``` + +- **`GetResponseStream(string command, string model = null, ResponseFormats responseFormat = ResponseFormats.NotSet)`** + - Streams responses from OpenAI for a command. + - **Usage Example:** + ```csharp + var api = new Api("your-api-key"); + await foreach (var chunk in api.GetResponseStream("Tell me a joke")) + { + Console.Write(chunk); + } + ``` + +- **`GetTransformationResponse(IEnumerable steps, ResponseFormats responseFormat = ResponseFormats.JsonObject)`** + - Executes a series of transformation steps, feeding each result into the next. + - **Usage Example:** + ```csharp + var api = new Api("your-api-key"); + var steps = new[] { "Write a greeting", "Format as JSON: #CURRENT_RESULT#" }; + var result = await api.GetTransformationResponse(steps); + Console.WriteLine(result); // e.g., "{\"greeting\":\"Hello!\"}" + ``` + +- **`GenerateDalleImage(string prompt, string model = "dall-e-3", Dictionary parameters = null, bool saveToS3 = true)`** + - Generates an image using DALL-E and optionally saves it to S3. + - **Usage Example:** + ```csharp + var api = new Api("your-api-key"); + var url = await api.GenerateDalleImage("A dog in space"); + Console.WriteLine(url); // S3 URL or temporary OpenAI URL + ``` + +--- + +## AssistantApi + +### AssistantApi Overview + +The `AssistantApi` class manages OpenAI Assistants, supporting creation, editing, deletion, and thread/message operations for both v1 and v2 API versions. + +### AssistantApi Key Methods + +- **`CreateNewAssistant(OpenAiCreateAssistantDto assistantDto)`** + - Creates a new v1 assistant. + - **Usage Example:** + ```csharp + var api = new AssistantApi("your-api-key"); + var assistant = new OpenAiCreateAssistantDto { Name = "Bot", Model = "gpt-3.5-turbo" }; + var assistantId = await api.CreateNewAssistant(assistant); + Console.WriteLine(assistantId); + ``` + +- **`EditAssistant(string assistantId, OpenAiCreateAssistantDto assistantDto)`** + - Edits an existing assistant (v1). + - **Usage Example:** + ```csharp + var api = new AssistantApi("your-api-key"); + var assistantDto = new OpenAiCreateAssistantDto { Name = "UpdatedBot" }; + var updatedId = await api.EditAssistant("assistant-id", assistantDto); + ``` + +- **`CreateNewThread(List messages = null)`** + - Creates a new thread with optional initial messages. + - **Usage Example:** + ```csharp + var api = new AssistantApi("your-api-key"); + var threadId = await api.CreateNewThread(new List { new ChatMessage("user", "Hi") }); + ``` + +- **`AddMessageToThread(ChatMessage message, string threadId)`** + - Adds a message to a thread. + - **Usage Example:** + ```csharp + var api = new AssistantApi("your-api-key"); + var messageId = await api.AddMessageToThread(new ChatMessage("user", "Hello"), "thread-id"); + ``` + +- **`CreateNewRun(string assistantId, string threadId)`** + - Creates a new run for an assistant on a thread. + - **Usage Example:** + ```csharp + var api = new AssistantApi("your-api-key"); + var runId = await api.CreateNewRun("assistant-id", "thread-id"); + ``` + +- **`GetRunStatus(string threadId, string runId)`** + - Retrieves the status of a run. + - **Usage Example:** + ```csharp + var api = new AssistantApi("your-api-key"); + var status = await api.GetRunStatus("thread-id", "run-id"); + Console.WriteLine(status); + ``` + +- **`GetThreadMessages(string threadId, string lastMessageId)`** + - Retrieves messages from a thread after a specified message ID. + - **Usage Example:** + ```csharp + var api = new AssistantApi("your-api-key"); + var messages = await api.GetThreadMessages("thread-id", "last-message-id"); + foreach (var msg in messages) Console.WriteLine(msg.Content); + ``` + +--- + +## Command + +### Command Overview + +The `Command` class builds structured prompts for OpenAI by allowing customization of context, tone, format, and more, generating a human-readable string representation. + +### Command Key Methods + +- **`Command(string request)`** + - Initializes a command with a base request. + - **Usage Example:** + ```csharp + var command = new Command("Write a story"); + ``` + +- **`Context(string context)`** + - Adds context to the command. + - **Usage Example:** + ```csharp + var command = new Command("Write a story").Context("Set in a futuristic city"); + ``` + +- **`Tone(params Tone[] tones)`** + - Specifies the tone(s) for the response. + - **Usage Example:** + ```csharp + var command = new Command("Explain AI").Tone(Tone.Formal); + ``` + +- **`Format(Format format)`** + - Sets the response format. + - **Usage Example:** + ```csharp + var command = new Command("List benefits").Format(Format.BulletPoints); + ``` + +- **`ToString()`** + - Generates the full command string. + - **Usage Example:** + ```csharp + var command = new Command("Write a story") + .Context("Set in a futuristic city") + .Tone(Tone.Narrative) + .Format(Format.Letter); + Console.WriteLine(command.ToString()); + // Outputs: "Context: Set in a futuristic city\nTone: narrative\nFormat: story\nWrite a story" + ``` + +--- + +## Examples +The quality of what you get from ChatGPT depends entirely on the quality of your prompt. +In the above basic example, the response is most likely not what you expect. +What you should do is to provide more details about the context and expectations. + +To help you with that, this Olive component provides the `Command` object, helping you get specific. + +### Example 1 +The following example, results in a single phrase (e.g. the name of the person) being returned, as opposed to a longer description: + +```csharp +using Olive.Gpt; +... + +var command = new Command("Who invented the computer mouse?") + .Length(Length.Phrase); + +string response = await gpt.GetResponse(command); +``` + +### Example 2 +The following example results in an essay with the following self-explanatory characteristics: + +```csharp + +var command = new Command("Who invented the computer mouse?") + .Context("Academic research homework about the history of the invention.") + .Audience("First-year computer science students") + .Length(Length.TwoPages) + .Format(Format.Essay) + .Tone(Tone.Formal, Tone.Narrative) + .Include(SupportingComponent.MultiplePerspectives, SupportingComponent.Citations); + +string response = await gpt.GetResponse(command); +``` + + +### Example 3 +The following example is also self-explanatory. It will result in a very different outcome. + +```csharp + +var command = new Command("Who invented the computer mouse?") + .Length(Length.ShortParagraph) + .Format(Format.Poem) + .Tone(Tone.Sarcastic, Tone.Humorous); + +string response = await gpt.GetResponse(command); +``` + +### What you can configure: + +You can specify: + +- **Context** - Provide background information, data, or context for accurate content generation. +- **Objective** - State the goal or purpose of the response (e.g., inform, persuade, entertain). +- **Scope** - Define the scope or range of the topic. +- **Audience** - Specify the target audience for tailored content. +- **Examples** - Provide examples of desired style, structure, or content. +- **ActAs** - Indicate a role or perspective to adopt(e.g., expert, critic, enthusiast). + +You can also easily pick and choose: + +- **Tone**: Formal, Informal, Persuasive, Descriptive, Expository, Narrative, Conversational, Sarcastic, Humorous, Emotional, Inspirational, Technical, Poetic, Slang, Colloquial, Euphemistic, Diplomatic, Authoritative, Didactic, Ironic, Cynical, Empathetic, Enthusiastic, Rhetorical, Objective, Subjective, Dismissive, Pessimistic, Optimistic +- **Length**: Phrase, Sentence, ShortParagraph, LongParagraph, TwoParagraphs, Page, TwoPages, FivePages +- **Format**: Essay, BulletPoints, Outline, Dialogue, List, Table, Flowchart, Definition, FAQs, Narrative, Poem, Anecdote, Script, NewsArticle, Review, Letter, Memo, Proposal, SocialMediaPost +- **Supporting Components**: Analogies, PreAddressCounterArguments, MultiplePerspectives, Citations, Quotes, Statistics + +--- + +### Notes +- **API Key:** The OpenAI API key must be provided when instantiating `Api` or `AssistantApi` (e.g., `new Api("your-api-key")`). It is not sourced from configuration. +- **S3 Integration:** If `saveToS3` is `true` in `GenerateDalleImage`, S3 settings are required. If not provided, the method falls back to returning a temporary OpenAI URL. \ No newline at end of file