Cloudflare Docs
Logs
Cloudflare Docs
Logs
GitHub icon
Edit this page on GitHub
Set theme to dark (⇧+D)
  1. Products
  2. Logs
  3. ...
  4. Enable destinations
  5. Enable S3-compatible endpoints

Enable Logpush to S3-compatible endpoints

Cloudflare Logpush supports pushing logs to S3-compatible destinations via the Cloudflare dashboard or via API, including:

For more information about Logpush and the current production APIs, refer to Cloudflare Logpush documentation.

​​ Manage via the Cloudflare dashboard

Enable Logpush to an S3-compatible destination via the dashboard.

To enable the Cloudflare Logpush service:

  1. Log in to the Cloudflare dashboard.

  2. Select the Enterprise account or domain you want to use with Logpush.

  3. Go to Analytics & Logs > Logs.

  4. Select Add Logpush job.

  5. In Select data set, choose the dataset to push to a storage service, and select Next.

  6. In Select data fields:

    • Select the data fields to include in your logs. Add or remove fields later by modifying your settings in Logs > Logpush.
    • In Advanced Settings, you can change the Timestamp format (RFC3339(default),Unix, or UnixNano), Sampling rate and enable redaction for CVE-2021-44228.
    • Under Filters you can select the events to include and/or remove from your logs. For more information, refer to Filters. Not all datasets have this option available.

  1. In Select a destination, choose S3-Compatible.

  2. Enter or select the following destination information:

    • S3 Compatible Bucket Path
    • Bucket region
    • Access Key ID
    • Secret Access Key
    • Endpoint URL

  3. Select Validate access.

  4. Select Save and Start Pushing to finish enabling Logpush.

Once connected, Cloudflare lists the S3-compatible destination as a connected service under Logs > Logpush. Edit or remove connected services from here.

​​ Manage via API

To set up S3-compatible endpoints:

  1. Create a job with the appropriate endpoint URL and authentication parameters.
  2. Enable the job to begin pushing logs.
Ensure Log Share permissions are enabled, before attempting to read or configure a Logpush job. For more information refer to the Roles section.

​​ 1. Create a job

To create a job, make a POST request to the Logpush jobs endpoint with the following fields:

  • name (optional) - Use your domain name as the job name.
  • destination_conf - A log destination consisting of an endpoint name, bucket name, bucket path, region, access-key-id, and secret-access-key in the following string format:
"s3://<BUCKET_NAME>/<BUCKET_PATH>?region=<REGION>&access-key-id=<ACCESS_KEY_ID>&secret-access-key=<SECRET_ACCESS_KEY>&endpoint=<ENDPOINT_URL>"
  • dataset - The category of logs you want to receive. Refer to Log fields for the full list of supported datasets.
  • logpull_options (optional) - To configure fields, sample rate, and timestamp format, refer to API configuration options.

Example request using cURL:

curl -s -X POST \
https://api.cloudflare.com/client/v4/zones/<ZONE_ID>/logpush/jobs \

-d '{"name":"<DOMAIN_NAME>",
"destination_conf":"s3://<BUCKET_NAME>/<BUCKET_PATH>?region=<REGION>&access-key-id=<ACCESS_KEY_ID>&secret-access-key=<SECRET_ACCESS_KEY>&endpoint=<ENDPOINT_URL>", "logpull_options": "fields=ClientIP,ClientRequestHost,ClientRequestMethod,ClientRequestURI,EdgeEndTimestamp,EdgeResponseBytes,EdgeResponseStatus,EdgeStartTimestamp,RayID&timestamps=rfc3339", "dataset": "http_requests"}' | jq .

Response:

{
  "errors": [],
  "messages": [],
  "result": {
    "id": 100,
    "dataset": "http_requests",
    "enabled": false,
    "name": "<DOMAIN_NAME>",
    "logpull_options": "fields=ClientIP,ClientRequestHost,ClientRequestMethod,ClientRequestURI,EdgeEndTimestamp,EdgeResponseBytes,EdgeResponseStatus,EdgeStartTimestamp,RayID&timestamps=rfc3339",
    "destination_conf": "s3://<BUCKET_NAME>/<BUCKET_PATH>?region=<REGION>&access-key-id=<ACCESS_KEY_ID>&secret-access-key=<SECRET_ACCESS_KEY>&endpoint=<ENDPOINT_URL>",
    "last_complete": null,
    "last_error": null,
    "error_message": null
  },
  "success": true

}

​​ 2. Enable (update) a job

To enable a job, make a PUT request to the Logpush jobs endpoint. You will use the job ID returned from the previous step in the URL, and send {"enabled": true} in the request body.

Example request using cURL:

curl -s -X PUT \
https://api.cloudflare.com/client/v4/zones/<ZONE_ID>/logpush/jobs/100 -d'{"enabled":true}' | jq .

Response:

{
  "errors": [],
  "messages": [],
  "result": {
    "id": 100,
    "dataset": "http_requests",
    "enabled": true,
    "name": "<DOMAIN_NAME>",
    "logpull_options": "fields=ClientIP,ClientRequestHost,ClientRequestMethod,ClientRequestURI,EdgeEndTimestamp,EdgeResponseBytes,EdgeResponseStatus,EdgeStartTimestamp,RayID&timestamps=rfc3339",
    "destination_conf": "s3://<BUCKET_NAME>/<BUCKET_PATH>?region=<REGION>&access-key-id=<ACCESS_KEY_ID>&secret-access-key=<SECRET_ACCESS_KEY>&endpoint=<ENDPOINT_URL>",
    "last_complete": null,
    "last_error": null,
    "error_message": null
  },
  "success": true

}