Arduino not writing full NMEA sentence to SD-card file

Question

I'm currently building a small GPS-box that's supposed to track my location and write the full NMEA sentence to an sd card.
(I want to pars it afterwards at my pc)
I'm using the Arduino Nano and the NEO-6M GPS Module to get the data.

What works: getting the NMEA data from the module, writing to the SD-card.
Outputting the data to the serial output over Serial.write works fine.

Now I have the problem that it looks like the Arduino can't write the data fast enough to the SD-card and desyncs with the GPS module. This occasionally produces things like this: $G3,3,09,32,20,248,*4D

I have some ideas on how to fix this:
1. write the data faster
2. always wait till the data is fully written before acquiring the next fix
3. only write every second GPS fix

I tried to implement these but failed every time (sorry I'm new to this).

Here is my current code:

#include <SoftwareSerial.h>
#include <SPI.h>
#include <SD.h>
SoftwareSerial GPS_Serial(4, 3); // GPS Module’s TX to D4 & RX to D3
File GPS_File;
int NBR = 1; //file number
void setup() {
 Serial.begin(9600);
 GPS_Serial.begin(9600);
 SD.begin(5); //SD Pin 
 //write data to a new file
 bool rn = false;
 while (rn == false) {
 if (SD.exists(String(NBR) + ".txt")) {
 NBR = NBR + 1;
 }
 else {
 GPS_File = SD.open(String(NBR) + ".txt", FILE_WRITE);
 GPS_File.write("START\n");
 GPS_File.close();
 rn = true;
 }
 }
}
int x = 0;
int temp = 0;
char gpsData;
char buff[300];
void loop() {
 //collect data and write to buffer
 while (GPS_Serial.available() > 0) {
 gpsData = GPS_Serial.read();
 x++;
 buff[x] = gpsData;
 }
 temp = x;
 x = 1;
 GPS_File = SD.open(String(NBR) + ".txt", FILE_WRITE);
 //copy from buffer to sd
 while (x <= temp) {
 GPS_File.print(buff[x]);
 x++;
 }
 GPS_File.close();
 x = 0;
 temp = 0;
}

• Majenko: You need to overhaul your serial reading. And don't keep closing and opening the file (use GPS_File.flush() instead). majenko.co.uk/blog/reading-serial-arduino
• the busybee: while (GPS_Serial.available() > 0) might break in the middle because the loop is faster than the speed of (byte) transmission. You might like to wait for the end-of-line character.
• DerEntinator: @Majenko implementing the File.flush() improved it a bit and made it possible to get full sets while the GPS has no signal. Sadly, when the GPS acquires a signal the data amount goes up by a lot and it can't keep up anymore leading to desyncs
• DerEntinator: @Majenko implementing the readline from your link sadly didn't improve it and led to more desyncs.

Answer 1

After trying different approaches I decided to dumb it down to its most basic level.
Without any fancy coding or buffers, I'm now just writing the data directly to the SD-card and flushing every 15 seconds which has the risk to loose up to 15 seconds of data i.e. 15 GPS-fixes (1 per second) when cutting off power.

The only other time where a potential data loss can happen is observed when the program flushes the accumulated data to the SD. This doesn't happen every time though.

To pars the NMEA-sentences into usable data I use GPSBabel. It automatically ignores the broken lines. After converting to .gpx I view it with Google Earth.

This is the "finished" code:

#include <SoftwareSerial.h>
#include <SPI.h>
#include <SD.h>
SoftwareSerial GPS_Serial(4, 3); // GPS Module’s TX to D4 & RX to D3
File GPS_File;
int NBR = 1; //file number
unsigned long TimerA; //save timer
bool sw = false; //save timer switch
void setup() {
 Serial.begin(9600);
 GPS_Serial.begin(9600);
 SD.begin(5); //SD Pin
 //write data to a new file
 bool rn = false;
 while (rn == false) {
 if (SD.exists(String(NBR) + ".txt")) {
 NBR = NBR + 1;
 }
 else {
 GPS_File = SD.open(String(NBR) + ".txt", FILE_WRITE);
 GPS_File.write("START\n");
 rn = true;
 }
 }
}
void loop() {
 while (GPS_Serial.available() > 0) {
 GPS_File.write(GPS_Serial.read());
 }
 //set up timer
 if ( sw == false) {
 TimerA = millis();
 sw = true;
 }
 //save every 15 seconds
 if (millis() - TimerA >= 15000UL) {
 GPS_File.flush();
 sw = false;
 }
}