How do I avoid text field truncation when saving a table join?

Question

Using QGIS I've joined tabular text data (all in fields of less than 255 characters) to a shapefile via a CSV file. All looks good in the joined table but when I save to a new shapefile to make the join permanent the long text fields get truncated to between 50-100 characters. How can I avoid this? It seems the problem is in the saving/exporting process since the initial join is fine.

I'm running QGIS 1.7.0 on Mac 10.6.8

Thanks, Matt

• Mapperz: add a 'null' row to your table with dummy text that contains 100 characters - export then delete that row to the correct result.
• Matt: Interesting, I added a null row with text and exported. The null row was not truncated but everything else still was. Any other thoughts? - Matt

Answer 1

The problem lies with the OGR library used by QGIS to read the data from the CSV file. It does not attempt to automatically determine the datatype and length of each field in the file like some software, instead relying on an accompanying .csvt file described in the OGR CSV page.

In the absence of a .csvt file, my guess is that OGR reports string field sizes as a default of 80 characters meaning that when a shapefile is saved after joining to a CSV, the shapefile's fields corresponding to those derived from the CSV will be set to this length, and text values longer than this will be truncated.

If a .csvt file is created for the CSV file to be joined then the field sizes are correctly read and passed through to the shapefile following joining, and text truncation is avoided.

(Tested on QGIS 1.8.0)

• user18674: It didn´t work for me. I have the csvt version for the csv file, and after joining the text values are still truncated. I also create a csv with the same name of the new shp, but it didn`t work either.