Skip to content

pgx v4 -> v5#247

Open
eldang wants to merge 14 commits into
CrunchyData:masterfrom
eldang:pgproto3-replacement
Open

pgx v4 -> v5#247
eldang wants to merge 14 commits into
CrunchyData:masterfrom
eldang:pgproto3-replacement

Conversation

@eldang

@eldang eldang commented May 18, 2026

Copy link
Copy Markdown
Contributor

After #246 I realised that https://nvd.nist.gov/vuln/detail/CVE-2026-32286 was still being flagged. It turns out there's no plan to fix that vulnerability in the older versions of pgproto3 because that module has since been rolled into pgx v5, and the maintainers understandably only want to maintain the current version. But the major version increment dropped a couple of structures that pg_tileserv was using. This PR is my attempt at replacing that functionality.

It's my first time making non-trivial changes to internal code for this tool, so I think it is worth more careful review than my usual minor dependency version updates. I will try to flag everything I have any uncertainty about....

Testing I've done so far:

  • Builds without errors
  • Passes go lint and go vet without any complaints
  • go test -v
  • Actually using the tile server for real in a project

Comment thread db.go
Comment on lines -50 to -53
config.ConnConfig.Logger = logrusadapter.NewLogger(log.New())
levelString, _ := (log.GetLevel() - 1).MarshalText()
pgxLevel, _ := pgx.LogLevelFromString(string(levelString))
config.ConnConfig.LogLevel = pgxLevel

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest this is the change I'm most unhappy about. I haven't been able to figure out how to use logrus with pgx v5. Removing this code entirely is working fine for my purposes, but I'm guessing there are log levels at which it may prevent us from getting the intended level of output.

Comment thread go.mod
Comment on lines -12 to +10
github.com/jackc/pgconn v1.14.3
github.com/jackc/pgtype v1.14.4
github.com/jackc/pgx/v4 v4.18.3
github.com/jackc/pgx/v5 v5.9.2

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pgconn and pgtype have been moved into pgx for v5.

Comment thread layer_table.go

// Failed to get estimate? Get the exact bounds.
if xmin.Status == pgtype.Null {
if !xmin.Valid {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pgtype v5 has switched the definition of Float8 (the type of each number in the bounding box) to:

type Float8 struct {
	Float64 float64
	Valid   bool
}

So I made two corresponding changes:

  • Getting the values out as .Float64 (above)
  • In the absence of a Status property, taking a guess that I could get the same result by checking for .Valid. I'm not certain of how fully equivalent this is, but it seems to be working the same.

Comment thread layer_table.go
arrLen := atts.Dimensions[0].Length
arrStart := atts.Dimensions[0].LowerBound - 1
elmLen := atts.Dimensions[1].Length
if len(atts) > 0 {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm assuming, without having had a way of testing this directly, that without an explicit TextArray.Status I can rely on simply checking if there's anything in the array.

Comment thread layer_table.go
if len(atts) > 0 {
elmLen := 4
arrLen := len(atts) / elmLen
arrStart := 0

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the old version of pgtype figured out the length of individual elements while unmarshalling JSON to the text array. I couldn't figure out a good way to do that, so I've hard-coded it as 4 on the assumption that the SQL which generates the JSON:

		(
			SELECT array_agg(ARRAY[sa.attname, st.typname, coalesce(da.description,''), sa.attnum::text]::text[] ORDER BY sa.attnum)
			FROM pg_attribute sa
			JOIN pg_type st ON sa.atttypid = st.oid
			LEFT JOIN pg_description da ON (c.oid = da.objoid and sa.attnum = da.objsubid)
			WHERE sa.attrelid = c.oid
			AND sa.attnum > 0
			AND NOT sa.attisdropped
			AND st.typname NOT IN ('geometry', 'geography')
		) AS props

can only ever make arrays in which the inner element always has 4 items. In doing this, I take some comfort from the hard-coded 1, 2, 3 offsets in the next few lines (to find the type, description, and ordering). But if my assumption is ever wrong, it is likely to be a crashing bug.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update to this comment: I realised late that I could actually make a 2-dimensional array of pgtype.Text values, which let me simplify the code a bit, and at least won't crash if we ever get > 4 elements inside a props value.

Comment thread util.go
// NewMetricsResponseWriter instantiates and returns a metricsResponseWriter
func NewMetricsResponseWriter(w http.ResponseWriter) *metricsResponseWriter {
// newMetricsResponseWriter instantiates and returns a metricsResponseWriter
func newMetricsResponseWriter(w http.ResponseWriter) *metricsResponseWriter {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was simply to silence a go vet warning. It didn't like that an exported function returns an unexported type. When I realised that the function is only ever called within this file, I figured simply making the function unexported would be a safe way to address that.

@eldang

eldang commented May 18, 2026

Copy link
Copy Markdown
Contributor Author

Unfortunately I can't see what the failing tests are. If the content of them is stuff I could potentially address, just let me know.

Comment thread layer_table.go
srid int
geometryType, idColumn string
atts pgtype.TextArray
atts [][]pgtype.Text

@eldang eldang May 19, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is where I'm really out of my comfort zone. pgtype v5 doesn't have a TextArray type at all, and it looks like that type used to do some automated array dimensions calculations for us. So I've had to replace the functionality that used to be provided by the Status and Dimensions members of TextArray, while replacing the actual text array member directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant