Invalid Input Error: Mismatch between the schema of different files

How to fix mismatch schema error when using read_csv_auto.

If you get an error saying "Invalid Input Error: Mismatch..." when running code similar to the above, you have a few options:

Delete any empty files

Empty files cause a mismatch error. Search all directories for empty files and delete them.

Debug - By filtering which files get loaded. Either:

  • Move files in/out of the loading directory until it works to narrow down which file is causing the problem.
  • Use the rejects_table feature to record which records are failing to load.
    rejects_table = 'rejects_table', ignore_errors = true