Error: Connector reply error: SQL##f - SqlState: XX000, ErrorCode: 30, ErrorMsg: [Amazon][Amazon Redshift] (30) Error occurred while trying to execute a query: [SQLState XX000] ERROR: write failedArticle Number: 000054583 | Last Modified: 2018/11/21
When fetching a large dataset with a load script containing Join clauses from Amazon Redshift.
Error: Connector reply error: SQL##f - SqlState: XX000 is related to either disk full or memory allocation issues. Multiple results on troubleshooting sites (stackoverflow, etc) state that Redshift has a VACUUM process to clean up rapidly changing data tables. Additionally, searching these errors online indicates a potentially massive overhead for join clauses. Example: https://dzone.com/articles/how-to-debug-disk-full-errors-in-redshift
• Join at Your Own Risk
• If the query that’s failing has a join clause, there’s a good chance that’s what’s causing your errors. When Redshift executes a join, it has a few strategies for connecting rows from different tables together. By default, it performs a “hash join” by creating hashes of the join key in each table, and then it distributes them to each other node in the cluster. That means each node will have to store hashes for every row of the table. When joining large tables, this quickly fills up disk space.
The expectation is that this issue will be more prevalent if there are a large amount of rows being pulled.
Have a Question?
Search Qlik's Support Knowledge database or request assisted support for highly complex issues.Submit a case
Experiencing a serious issue, please contact us by phone. View phone numbers and hours by region.