Merge pull request #4 from pascalbaljet/report-query-callback

Optional callback to filter queries that are reported in the `QueryLogWritten` event

Author: onlime

.github/workflows/ci.yml

@@ -8,17 +8,14 @@ jobs:
     strategy:
       fail-fast: true
       matrix:
-        php: [8.3, 8.2, 8.1]
+        php: [8.3, 8.2]
         laravel: [11.*, 10.*]
         stability: [prefer-lowest, prefer-stable]
         include:
           - laravel: 11.*
             testbench: 9.*
           - laravel: 10.*
             testbench: 8.*
-        exclude:
-          - laravel: 11.*
-            php: 8.1
 
     name: P${{ matrix.php }} - L${{ matrix.laravel }} - ${{ matrix.stability }} - ${{ matrix.os }}
 

README.md

@@ -19,16 +19,16 @@ It reports a lot of metadata like total query count, total execution time, origi
    ```bash
    $ composer require onlime/laravel-sql-reporter --dev
    ```
-   in console to install this module (Notice `--dev` flag - it's recommended to use this package only for development). 
+   in console to install this module (Notice `--dev` flag - it's recommended to use this package only for development).
 
    Laravel uses package auto-discovery, and it will automatically load this service provider, so you don't need to add anything into the `providers` section of `config/app.php`.
-   
+
 2. Run the following in your console to publish the default configuration file:
-  
+
     ```bash
     $ php artisan vendor:publish --provider="Onlime\LaravelSqlReporter\Providers\ServiceProvider"
     ```
-    
+
     By default, you should not edit published file because all the settings are loaded from `.env` file by default.
 
 3. In your `.env` file add the following entries:
@@ -48,7 +48,7 @@ It reports a lot of metadata like total query count, total execution time, origi
     SQL_REPORTER_FORMAT_HEADER_FIELDS="origin,datetime,status,user,env,agent,ip,host,referer"
     SQL_REPORTER_FORMAT_ENTRY_FORMAT="-- Query [query_nr] [[query_time]]\\n[query]"
     ```
-    
+
     and adjust values to your needs. You can skip variables for which you want to use default values.
 
     To only log DML / modifying queries like `INSERT`, `UPDATE`, `DELETE`, but not logging any updates on
@@ -58,17 +58,19 @@ It reports a lot of metadata like total query count, total execution time, origi
    SQL_REPORTER_QUERIES_INCLUDE_PATTERN="/^(?!SELECT).*/i"
    SQL_REPORTER_QUERIES_EXCLUDE_PATTERN="/^UPDATE.*(last_visit|remember_token)/i"
    ```
-   
+
     If you have also `.env.example` it's recommended to add those entries also in `.env.example` file just to make sure everyone knows about those env variables. Be aware that `SQL_REPORTER_DIRECTORY` is directory inside storage directory.
-   
+
     To find out more about those setting please take a look at [Configuration file](config/sql-reporter.php)
-   
+
 4. Make sure directory specified in `.env` file exists in storage path, and you have valid permissions to create and modify files in this directory (If it does not exist this package will automatically create it when needed, but it's recommended to create it manually with valid file permissions)
 
 5. Make sure on live server you will set logging SQL queries to false in your `.env` file: `SQL_REPORTER_QUERIES_ENABLED=false`. This package is recommended to be used only for development to not impact production application performance.
 
 ## Optional
 
+### GeoIP support
+
 For optional GeoIP support (adding country information to client IP in log headers), you may install [stevebauman/location](https://github.com/stevebauman/location) in your project:
 
 ```bash
@@ -78,6 +80,24 @@ $ php artisan vendor:publish --provider="Stevebauman\Location\LocationServicePro
 
 It will be auto-detected, no configuration needed for this. If you wish to use a different driver than the default [IpApi](https://ip-api.com/), e.g. `MaxMind` make sure you correctly configure it according to the docs: [Available Drivers](https://github.com/stevebauman/location#available-drivers)
 
+### `QueryLogWritten` event
+
+This package fires a `QueryLogWritten` event after the log file has been written. You may use this event to further debug or analyze the logged queries in your application. The queries are filtered by the `SQL_REPORTER_QUERIES_REPORT_PATTERN` setting, which comes with a sensible default to exclude `SELECT` queries and some default tables like `sessions`, `jobs`, `bans`, `logins`. If you don't want to filter any queries, you may leave this setting empty.
+
+In addition to the pattern, you may also configure a callback to define your own custom filtering logic, for example, in your `AppServiceProvider`:
+
+```php
+use Onlime\LaravelSqlReporter\SqlQuery;
+use Onlime\LaravelSqlReporter\Writer;
+
+Writer::shouldReportQuery(function (SqlQuery $query) {
+    // Only include queries in the `QueryLogWritten` event that took longer than 100ms
+    return $query->time > 100;
+});
+```
+
+With the `SqlQuery` object, you have access to both `$rawQuery` and the (unprepared) `$query`/`$bindings`. The filter possibilities by providing a callback to `Writer::shouldReportQuery()` are endless!
+
 ## Development
 
 Checkout project and run tests:
@@ -87,10 +107,10 @@ $ git clone https://github.com/onlime/laravel-sql-reporter.git
 $ cd laravel-sql-reporter
 $ composer install
 
-# run unit tests
-$ vendor/bin/phpunit
+# run both Feature and Unit tests
+$ vendor/bin/pest
 # run unit tests with coverage report
-$ XDEBUG_MODE=coverage vendor/bin/phpunit
+$ vendor/bin/pest --coverage
 ```
 
 ## FAQ
@@ -101,7 +121,7 @@ This package was inspired by [mnabialek/laravel-sql-logger](https://github.com/m
 
 - Query logging is not triggered upon each query execution but instead at a final step, using `RequestHandled` and `CommandFinished` events.
 - This allows us to include much more information about the whole query executions like total query count, total execution time, and very detailed header information like origin (request URL/console command), authenticated user, app environment, client browser agent / IP / hostname.
-- This package is greatly simplified and only provides support for Laravel 10+ / PHP 8.1+
+- This package is greatly simplified and only provides support for Laravel 10+ / PHP 8.2+
 - It uses the Laravel built-in query logging (`DB::enableQueryLog()`) which logs all queries in memory, which should perform much better than writing every single query to the log file.
 - By default, `onlime/laravel-sql-reporter` produces much nicer log output, especially since we only write header information before the first query.
 
@@ -152,8 +172,8 @@ All changes are listed in [CHANGELOG](CHANGELOG.md)
 
 ## Caveats
 
-- If your application crashes, this package will not log any queries, as logging is only triggered at the end. As alternative, you could use [mnabialek/laravel-sql-logger](https://github.com/mnabialek/laravel-sql-logger) which triggers sql logging on each query execution.
-- It's currently not possible to log slow queries into a separate logfile. I wanted to keep that package simpel.
+- If your application crashes, this package will not log any queries, as logging is only triggered at the end of the request cycle. As alternative, you could use [mnabialek/laravel-sql-logger](https://github.com/mnabialek/laravel-sql-logger) which triggers sql logging on each query execution.
+- It's currently not possible to log slow queries into a separate logfile. I wanted to keep that package simple.
 
 ## TODO
 

composer.json

@@ -14,7 +14,7 @@
         }
     ],
     "require": {
-        "php": "^8.1",
+        "php": "^8.2",
         "illuminate/support": "^10.15|^11.0",
         "illuminate/filesystem": "^10.15|^11.0",
         "illuminate/container": "^10.15|^11.0"

src/Config.php

@@ -88,7 +88,7 @@ public function queriesExcludePattern(): string
      */
     public function queriesReportPattern(): string
     {
-        return $this->repository->get('sql-reporter.queries.report_pattern');
+        return $this->repository->get('sql-reporter.queries.report_pattern') ?: '';
     }
 
     /**

src/Formatter.php

@@ -23,9 +23,9 @@ public function __construct(
     public function getLine(SqlQuery $query): string
     {
         $replace = [
-            '[query_nr]'   => $query->number(),
+            '[query_nr]'   => $query->number,
             '[datetime]'   => Carbon::now()->toDateTimeString(),
-            '[query_time]' => $this->time($query->time()),
+            '[query_time]' => $this->time($query->time),
             '[query]'      => $this->getQueryLine($query),
             '[separator]'  => $this->separatorLine(),
             '\n'           => PHP_EOL,
@@ -115,7 +115,7 @@ protected function originLine(): string
      */
     protected function getQueryLine(SqlQuery $query): string
     {
-        return $query->rawQuery().';';
+        return $query->rawQuery.';';
     }
 
     /**

src/Providers/SqlReporterServiceProvider.php

@@ -17,7 +17,7 @@ class SqlReporterServiceProvider extends ServiceProvider
     /**
      * {@inheritdoc}
      */
-    public function register()
+    public function register(): void
     {
         $this->config = $this->app->make(Config::class);
 
@@ -31,7 +31,7 @@ public function register()
     /**
      * {@inheritdoc}
      */
-    public function boot()
+    public function boot(): void
     {
         $this->publishes([
             $this->configFileLocation() => config_path('sql-reporter.php'),

src/SqlLogger.php

@@ -2,15 +2,11 @@
 
 namespace Onlime\LaravelSqlReporter;
 
+use Illuminate\Support\Collection;
 use Illuminate\Support\Facades\DB;
 
 class SqlLogger
 {
-    /**
-     * Number of executed queries.
-     */
-    private int $queryNumber = 0;
-
     /**
      * SqlLogger constructor.
      */
@@ -24,11 +20,22 @@ public function __construct(
      */
     public function log(): void
     {
-        foreach (DB::getRawQueryLog() as $query) {
-            $this->writer->writeQuery(
-                new SqlQuery(++$this->queryNumber, $query['raw_query'], $query['time'])
-            );
-        }
+        $queryLog = DB::getQueryLog();
+
+        // getQueryLog() and getRawQueryLog() have the same keys
+        // see \Illuminate\Database\Connection::getRawQueryLog()
+        Collection::make(DB::getRawQueryLog())
+            ->map(fn (array $query, int $key) => new SqlQuery(
+                $key + 1,
+                $query['raw_query'],
+                $query['time'],
+                $queryLog[$key]['query'],
+                $queryLog[$key]['bindings']
+            ))
+            ->each(function (SqlQuery $query) {
+                $this->writer->writeQuery($query);
+            });
+
         $this->writer->report();
     }
 }

src/SqlQuery.php

@@ -2,36 +2,24 @@
 
 namespace Onlime\LaravelSqlReporter;
 
-class SqlQuery
+readonly class SqlQuery
 {
     public function __construct(
-        private int $number,
-        private string $rawQuery,
-        private float $time
+        public int $number,
+        public string $rawQuery,
+        public float $time,
+        public string $query,
+        public array $bindings = []
     ) {
     }
 
-    /**
-     * Get query number.
-     */
-    public function number(): int
-    {
-        return $this->number;
-    }
-
-    /**
-     * Get raw SQL query with embedded bindings.
-     */
-    public function rawQuery(): string
-    {
-        return $this->rawQuery;
-    }
-
-    /**
-     * Get query execution time.
-     */
-    public function time(): float
-    {
-        return $this->time;
+    public static function make(
+        int $number,
+        string $rawQuery,
+        float $time,
+        ?string $query = null,
+        array $bindings = []
+    ): self {
+        return new self($number, $rawQuery, $time, $query ?? $rawQuery, $bindings);
     }
 }

src/Writer.php

@@ -2,6 +2,7 @@
 
 namespace Onlime\LaravelSqlReporter;
 
+use Closure;
 use Onlime\LaravelSqlReporter\Events\QueryLogWritten;
 
 class Writer
@@ -22,21 +23,34 @@ class Writer
      */
     private array $reportQueries = [];
 
+    /**
+     * Callback to determine whether query should be reported.
+     */
+    private static ?Closure $shouldReportQuery = null;
+
     public function __construct(
-        private Formatter $formatter,
-        private Config $config,
-        private FileName $fileName
+        private readonly Formatter $formatter,
+        private readonly Config $config,
+        private readonly FileName $fileName
     ) {
     }
 
+    /**
+     * Set callback to determine whether query should be reported.
+     */
+    public static function shouldReportQuery(callable $callback): void
+    {
+        self::$shouldReportQuery = $callback;
+    }
+
     /**
      * Write a query to log.
      *
      * @return bool true if query was written to log, false if skipped
      */
     public function writeQuery(SqlQuery $query): bool
     {
-        $this->createDirectoryIfNotExists($query->number());
+        $this->createDirectoryIfNotExists($query->number);
 
         if ($this->shouldLogQuery($query)) {
             if ($this->loggedQueryCount === 0) {
@@ -60,9 +74,15 @@ public function writeQuery(SqlQuery $query): bool
     /**
      * Verify whether query should be reported.
      */
-    private function shouldReportSqlQuery(SqlQuery $query): bool
+    public function shouldReportSqlQuery(SqlQuery $query): bool
     {
-        return preg_match($this->config->queriesReportPattern(), $query->rawQuery()) === 1;
+        $pattern = $this->config->queriesReportPattern();
+
+        if ($pattern && preg_match($pattern, $query->rawQuery) !== 1) {
+            return false;
+        }
+
+        return call_user_func(self::$shouldReportQuery ?? fn () => true, $query);
     }
 
     /**
@@ -92,9 +112,9 @@ protected function directory(): string
     protected function shouldLogQuery(SqlQuery $query): bool
     {
         return $this->config->queriesEnabled() &&
-            $query->time() >= $this->config->queriesMinExecTime() &&
-            preg_match($this->config->queriesIncludePattern(), $query->rawQuery()) &&
-            ! preg_match($this->config->queriesExcludePattern(), $query->rawQuery());
+            $query->time >= $this->config->queriesMinExecTime() &&
+            preg_match($this->config->queriesIncludePattern(), $query->rawQuery) &&
+            ! preg_match($this->config->queriesExcludePattern(), $query->rawQuery);
     }
 
     /**