Skip to content

fix: out of memory errors for large number of tables #4523

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
taimoorzaeem wants to merge 1 commit into from
Closed

fix: out of memory errors for large number of tables #4523

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ All notable changes to this project will be documented in this file. From versio

## Unreleased

### Fixed

- Fix out of memory errors for large number of tables by @taimoorzaeem in #4523
+ Disables the table not found error `hint` generation for more than 4000 tables

## [14.1] - 2025-11-05

## Fixed
Expand Down
5 changes: 4 additions & 1 deletion src/PostgREST/Error.hs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -428,9 +428,12 @@ noRpcHint schema procName params allProcs overloadedProcs =

-- |
-- Do a fuzzy search in all tables in the same schema and return closest result
-- We have a upper limit for hint generation as 4000 tables to avoid memory
-- and performance issues.
tableNotFoundHint :: Text -> Text -> [Table] -> Maybe Text
tableNotFoundHint schema tblName tblList
= fmap (\tbl -> "Perhaps you meant the table '" <> schema <> "." <> tbl <> "'") perhapsTable
| length tblList <= 4000 = fmap (\tbl -> "Perhaps you meant the table '" <> schema <> "." <> tbl <> "'") perhapsTable
Copy link
Contributor

@mkleczek mkleczek Dec 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are several problems with this change:

length tblList complexity is linear in the list size and it has to iterate the whole list even if it is longer than 4k.

Leaving it like this opens up a DoS vulnerability.

4k is too many from my tests. On my machine PostgREST became unstable with 1k tables.
This is another DoS vulnerability.

Re-creating FuzzySet upon each request opens up yet another DoS vulnerability.

I'm afraid there is really no other sensible solution to this than #4472

Copy link
Collaborator Author

@taimoorzaeem taimoorzaeem Dec 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4k is too many from my tests. On my machine PostgREST became unstable with 1k tables.

Yes, agreed. I was proposing 500 to be the upper limit as mentioned in #4523 (comment).

| otherwise = Nothing
where
perhapsTable = Fuzzy.getOne fuzzyTableSet tblName
fuzzyTableSet = Fuzzy.fromList [ tableName tbl | tbl <- tblList, tableSchema tbl == schema]
Expand Down