Skip to content

Add sequence_overflow metrics#1292

Open
alphabet5 wants to merge 1 commit into
prometheus-community:masterfrom
alphabet5:master
Open

Add sequence_overflow metrics#1292
alphabet5 wants to merge 1 commit into
prometheus-community:masterfrom
alphabet5:master

Conversation

@alphabet5

Copy link
Copy Markdown

This query was based on

https://www.crunchydata.com/blog/the-integer-at-the-end-of-the-universe-integer-overflow-in-postgres

and exposes metrics for % used for the underlying column type.

This requires usage on all sequences, or GRANT pg_read_all_data TO postgres_exporter;

pg_read_all_data is a bit excessive, but granting usage on all sequences won't apply for new sequences that are added, and setting default privileges for schemas won't apply if new schemas are added.

pg_sequence_overflow_column_ratio{column_datatype="bigint", datname="mydb", instance="mydb.example.com", job="postgres-exporter", owned_by="my_table.id", schemaname="public", sequence="my_table_id_seq", sequence_datatype="bigint"}
	8.745220514546834e-10
pg_sequence_overflow_last_value{column_datatype="bigint", datname="mydb", instance="mydb.example.com", job="postgres-exporter", owned_by="my_table.id", region="dal", schemaname="public", sequence="my_table_id_seq", sequence_datatype="bigint"}
	8066042235
pg_sequence_overflow_sequence_ratio{column_datatype="bigint", datname="mydb", instance="mydb.example.com", job="postgres-exporter", owned_by="my_table.id", schemaname="public", sequence="my_table_id_seq", sequence_datatype="bigint"}
	8.745220514546834e-10

This query was based on

https://www.crunchydata.com/blog/the-integer-at-the-end-of-the-universe-integer-overflow-in-postgres

and exposes metrics for % used for the underlying column type.

This requires usage on all sequences, or `GRANT pg_read_all_data TO postgres_exporter;`

pg_read_all_data is a bit excessive, but granting usage on all sequences won't apply for new sequences that are added, and setting default privileges for schemas won't apply if new schemas are added.

```text
pg_sequence_overflow_column_ratio{column_datatype="bigint", datname="mydb", instance="mydb.example.com", job="postgres-exporter", owned_by="my_table.id", schemaname="public", sequence="my_table_id_seq", sequence_datatype="bigint"}
	8.745220514546834e-10
pg_sequence_overflow_last_value{column_datatype="bigint", datname="mydb", instance="mydb.example.com", job="postgres-exporter", owned_by="my_table.id", region="dal", schemaname="public", sequence="my_table_id_seq", sequence_datatype="bigint"}
	8066042235
pg_sequence_overflow_sequence_ratio{column_datatype="bigint", datname="mydb", instance="mydb.example.com", job="postgres-exporter", owned_by="my_table.id", schemaname="public", sequence="my_table_id_seq", sequence_datatype="bigint"}
	8.745220514546834e-10
```

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new optional sequence_overflow collector to expose “% used” overflow-related metrics for owned sequences, based on the underlying sequence and column integer types.

Changes:

  • Introduces a new sequence_overflow collector (default disabled) that queries all non-template databases for owned sequences and exports last-value and ratio metrics.
  • Documents the new collector flag and required privileges in the README.
  • Adds unit tests validating metric emission for normal and NULL/unused sequence cases.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File Description
README.md Documents the new collector.sequence_overflow flag, default state, and required privileges.
collector/pg_sequence_overflow.go Implements the new collector, including DB discovery, per-DB querying, and metric descriptors.
collector/pg_sequence_overflow_test.go Adds unit tests for updateDatabase() metric output.
Comments suppressed due to low confidence (1)

collector/pg_sequence_overflow.go:110

  • There are no unit tests for dsnForDatabase, but it has tricky behavior around URL vs key=value DSNs and dbname precedence (path vs query param). Adding focused tests for common DSN forms (postgresql://.../db, postgresql://.../?dbname=..., and key=value DSN with existing/absent dbname) would help prevent regressions.
// dsnForDatabase returns a new DSN with the database name replaced.
// It updates both the URL path and the dbname query parameter (if present),
// since libpq gives precedence to the query parameter over the path.
func dsnForDatabase(dsn string, datname string) (string, error) {
	u, err := url.Parse(dsn)
	if err != nil {
		return "", fmt.Errorf("could not parse DSN: %w", err)
	}
	u.Path = "/" + datname
	if q := u.Query(); q.Get("dbname") != "" {
		q.Set("dbname", datname)
		u.RawQuery = q.Encode()
	}
	return u.String(), nil
}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +96 to +110
// dsnForDatabase returns a new DSN with the database name replaced.
// It updates both the URL path and the dbname query parameter (if present),
// since libpq gives precedence to the query parameter over the path.
func dsnForDatabase(dsn string, datname string) (string, error) {
u, err := url.Parse(dsn)
if err != nil {
return "", fmt.Errorf("could not parse DSN: %w", err)
}
u.Path = "/" + datname
if q := u.Query(); q.Get("dbname") != "" {
q.Set("dbname", datname)
u.RawQuery = q.Encode()
}
return u.String(), nil
}
Comment thread collector/pg_sequence_overflow.go
Comment on lines +67 to +84
sequenceOverflowQuery = `
SELECT
seqs.relname AS sequence,
format_type(s.seqtypid, NULL) AS sequence_datatype,
CONCAT(tbls.relname, '.', attrs.attname) AS owned_by,
format_type(attrs.atttypid, atttypmod) AS column_datatype,
ns.nspname AS schemaname,
COALESCE(pg_sequence_last_value(seqs.oid::regclass), 0) AS last_sequence_value,
COALESCE(CASE format_type(s.seqtypid, NULL)
WHEN 'smallint' THEN pg_sequence_last_value(seqs.oid::regclass) / 32767::float
WHEN 'integer' THEN pg_sequence_last_value(seqs.oid::regclass) / 2147483647::float
WHEN 'bigint' THEN pg_sequence_last_value(seqs.oid::regclass) / 9223372036854775807::float
END, 0) AS sequence_ratio,
COALESCE(CASE format_type(attrs.atttypid, NULL)
WHEN 'smallint' THEN pg_sequence_last_value(seqs.oid::regclass) / 32767::float
WHEN 'integer' THEN pg_sequence_last_value(seqs.oid::regclass) / 2147483647::float
WHEN 'bigint' THEN pg_sequence_last_value(seqs.oid::regclass) / 9223372036854775807::float
END, 0) AS column_ratio
Comment on lines +146 to +162
// Query sequence metrics from each database individually.
for _, datname := range databases {
newDSN, err := dsnForDatabase(instance.dsn, datname)
if err != nil {
c.log.Debug("Skipping database", "datname", datname, "err", err)
continue
}
dbConn, err := sql.Open("postgres", newDSN)
if err != nil {
c.log.Debug("Skipping database", "datname", datname, "err", err)
continue
}
if err := c.updateDatabase(ctx, dbConn, datname, ch); err != nil {
c.log.Debug("Skipping database", "datname", datname, "err", err)
}
dbConn.Close()
}

@sysadmind sysadmind left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The commit is also missing the DCO which is a requirement for this project. The CI details have instructions for how to fix it.

// dsnForDatabase returns a new DSN with the database name replaced.
// It updates both the URL path and the dbname query parameter (if present),
// since libpq gives precedence to the query parameter over the path.
func dsnForDatabase(dsn string, datname string) (string, error) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This functionality is unique to this collector. I think the intention is that these need to be run per database, but I think that other functionality would benefit from that too. I don't think that it should be specific to this collector. It's also not obvious that this collector would make new connections.

Comment on lines +76 to +78
WHEN 'smallint' THEN pg_sequence_last_value(seqs.oid::regclass) / 32767::float
WHEN 'integer' THEN pg_sequence_last_value(seqs.oid::regclass) / 2147483647::float
WHEN 'bigint' THEN pg_sequence_last_value(seqs.oid::regclass) / 9223372036854775807::float

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this exporter needs to calculate these values. That can be done in the Prometheus query. Having the raw value is much more useful for Prometheus. It means you can calculate this value on the fly and you know the absolute value, all with only storing a single time series.

Comment on lines +80 to +83
COALESCE(CASE format_type(attrs.atttypid, NULL)
WHEN 'smallint' THEN pg_sequence_last_value(seqs.oid::regclass) / 32767::float
WHEN 'integer' THEN pg_sequence_last_value(seqs.oid::regclass) / 2147483647::float
WHEN 'bigint' THEN pg_sequence_last_value(seqs.oid::regclass) / 9223372036854775807::float

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here as above - we can calculate this at Prometheus query time.

c.log.Debug("Skipping database", "datname", datname, "err", err)
continue
}
dbConn, err := sql.Open("postgres", newDSN)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with copilot - this creates a lot of new connections which might not be desirable for users.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants