Logflare
Logflare is a centralized web-based log management solution to easily access Cloudflare, Vercel & Elixir logs.
The Logflare Wrapper allows you to read data from Logflare endpoints within your Postgres database.
Preparation
Before you can query Logflare, you need to enable the Wrappers extension and store your credentials in Postgres.
Enable Wrappers
Make sure the wrappers extension is installed on your database:
_10create extension if not exists wrappers with schema extensions;
Enable the Logflare Wrapper
Enable the logflare_wrapper FDW:
_10create foreign data wrapper logflare_wrapper_10 handler logflare_fdw_handler_10 validator logflare_fdw_validator;
Store your credentials (optional)
By default, Postgres stores FDW credentials inside pg_catalog.pg_foreign_server in plain text. Anyone with access to this table will be able to view these credentials. Wrappers is designed to work with Vault, which provides an additional level of security for storing credentials. We recommend using Vault to store your credentials.
_10-- Save your Logflare API key in Vault and retrieve the `key_id`_10insert into vault.secrets (name, secret)_10values (_10 'logflare',_10 'YOUR_SECRET'_10)_10returning key_id;
Connecting to Logflare
We need to provide Postgres with the credentials to connect to Logflare, and any additional options. We can do this using the create server command:
_10create server logflare_server_10 foreign data wrapper logflare_wrapper_10 options (_10 api_key_id '<key_ID>' -- The Key ID from above._10 );
Create a schema
We recommend creating a schema to hold all the foreign tables:
_10create schema if not exists logflare;
Options
The full list of foreign table options are below:
endpoint- Logflare endpoint UUID or name, required.
Entities
Logflare
This is an object representing Logflare endpoint data.
Ref: Logflare docs
Operations
| Object | Select | Insert | Update | Delete | Truncate |
|---|---|---|---|---|---|
| Logflare | ✅ | ❌ | ❌ | ❌ | ❌ |
Usage
_10create foreign table logflare.my_logflare_table (_10 id bigint,_10 name text,_10 _result text_10)_10 server logflare_server_10 options (_10 endpoint '9dd9a6f6-8e9b-4fa4-b682-4f2f5cd99da3'_10 );
Notes
Meta Column _result:
- Data type must be
text - Stores the whole result record in JSON string format
- Use JSON queries to extract fields:
_result::json->>'field_name'
Query Parameters:
- Use parameter columns with prefix
_param_ - Example:
_param_org_id,_param_iso_timestamp_start - Parameters are passed to the Logflare endpoint
Query Pushdown Support
This FDW doesn't support query pushdown.
Limitations
This section describes important limitations and considerations when using this FDW:
- Full result sets are loaded into memory, which can impact PostgreSQL performance with large datasets
- Parameter names must be prefixed with 'param' and match the expected endpoint parameters exactly
- Materialized views using these foreign tables may fail during logical backups
Examples
Basic Example
Given a Logflare endpoint response:
_10[_10 {_10 "id": 123,_10 "name": "foo"_10 }_10]
You can create and query a foreign table:
_11create foreign table logflare.people (_11 id bigint,_11 name text,_11 _result text_11)_11 server logflare_server_11 options (_11 endpoint '9dd9a6f6-8e9b-4fa4-b682-4f2f5cd99da3'_11 );_11_11select * from logflare.people;
Query Parameters Example
For an endpoint accepting parameters:
- org_id
- iso_timestamp_start
- iso_timestamp_end
With response format:
_10[_10 {_10 "db_size": "large",_10 "org_id": "123",_10 "runtime_hours": 21.95,_10 "runtime_minutes": 1317_10 }_10]
Create and query the table with parameters:
_25create foreign table logflare.runtime_hours (_25 db_size text,_25 org_id text,_25 runtime_hours numeric,_25 runtime_minutes bigint,_25 _param_org_id bigint,_25 _param_iso_timestamp_start text,_25 _param_iso_timestamp_end text,_25 _result text_25)_25 server logflare_server_25 options (_25 endpoint 'my.custom.endpoint'_25 );_25_25select_25 db_size,_25 org_id,_25 runtime_hours,_25 runtime_minutes_25from_25 logflare.runtime_hours_25where _param_org_id = 123_25 and _param_iso_timestamp_start = '2023-07-01 02:03:04'_25 and _param_iso_timestamp_end = '2023-07-02';