StoreProhibited error while trying to read file to PROGMEM

Question

I'm trying to read a file from micro SD on an ESP32-cam board and upload to an FTP server, but when I try to read data from the opened file to an unsigned char, it gives me the following error:

Guru Meditation Error: Core 1 panic'ed (StoreProhibited). Exception was unhandled.
Core 1 register dump:
PC : 0x4008c674 PS : 0x00060030 A0 : 0x800ed1c7 A1 : 0x3ffb1e20 
A2 : 0x00000000 A3 : 0x3ffd7c78 A4 : 0x00000080 A5 : 0x00000000 
A6 : 0x474e5089 A7 : 0x0a1a0a0d A8 : 0x00000000 A9 : 0x3ffb1df0 
A10 : 0x00000080 A11 : 0x3ffb66e4 A12 : 0x3ffd7c78 A13 : 0x00000080 
A14 : 0x00002494 A15 : 0xff000000 SAR : 0x00000018 EXCCAUSE: 0x0000001d 
EXCVADDR: 0x00000000 LBEG : 0x4008c670 LEND : 0x4008c68c LCOUNT : 0x00000007 
ELF file SHA256: 0000000000000000
Backtrace: 0x4008c674:0x3ffb1e20 0x400ed1c4:0x3ffb1e30 0x400ed245:0x3ffb1e60 0x400d06e7:0x3ffb1e80 0x400d084f:0x3ffb1f20 0x400d2b5a:0x3ffb1fb0 0x4008efde:0x3ffb1fd0
Rebooting...

The code is the following:

// Wifi and FTP Client Lib
#include <WiFi.h>
#include <WiFiClient.h> 
#include "ESP32_FTPClient.h"
// MicroSD
#include "driver/sdmmc_host.h"
#include "driver/sdmmc_defs.h"
#include "sdmmc_cmd.h"
#include "esp_vfs_fat.h"
// FTP Server credentials
char ftp_server[] = "192.168.1.50";
char ftp_user[] = "username";
char ftp_pass[] = "password";
// wifi access
const char* ssid = "ssid";
const char* password = "psk";
// file to be read and uploaded
FILE *fileUpload = NULL;
// ftp client object
ESP32_FTPClient ftp (ftp_server, ftp_user, ftp_pass);
// micro sd initialization
static esp_err_t init_sdcard()
{
 esp_err_t ret = ESP_FAIL;
 sdmmc_host_t host = SDMMC_HOST_DEFAULT();
 sdmmc_slot_config_t slot_config = SDMMC_SLOT_CONFIG_DEFAULT();
 esp_vfs_fat_sdmmc_mount_config_t mount_config = {
 .format_if_mount_failed = false,
 .max_files = 10,
 };
 sdmmc_card_t *card;
 ret = esp_vfs_fat_sdmmc_mount("/sdcard", &host, &slot_config, &mount_config, &card);
}
void FTP_upload( void );
void FTP_upload()
{
 // initialize file name
 char fileName[100];
 sprintf(fileName, "/sdcard/img.png");
 fileUpload = fopen(fileName, "r+");
 
 // notify reading of the file
 Serial.print("Reading file: ");
 Serial.println(fileName);
 if (fileUpload == NULL) 
 {
 Serial.println("Cannot open file");
 return;
 }
 // retrieve file size
 fseek(fileUpload, 0, SEEK_END);
 long int size = ftell(fileUpload);
 fclose(fileUpload);
 // print file size to serial
 Serial.print("File size: ");
 Serial.println(size);
 
 // Reading data to array of unsigned chars
 fileUpload = fopen(fileName, "r+");
 unsigned char * in PROGMEM = (unsigned char *) malloc(size);
 
 // display allocated memory
 Serial.print("Allocated memory: ");
 Serial.println(sizeof(in));
 // read file to allocated memory
 int bytes_read = fread(in, sizeof(unsigned char), size, fileUpload); // <------- THIS ROW RISES THE ERROR
 
// close file
 fclose(fileUpload);
 // Open FTP connection
 Serial.println("Starting FTP upload...");
 ftp.OpenConnection();
 
 //Create a file and write the image data to it;
 ftp.InitFile("Type I");
 // change remote directory
 ftp.ChangeWorkDir("/Shared/");
 // specify remote file name
 String strNome = "img.png";
 // convert String to char
 const char *f_name = strNome.c_str();
 
 // create a new file with the given name inside the remote directory
 ftp.NewFile( f_name );
 // write data read above from local file to remote file
 ftp.WriteData(in, sizeof(in));
 // close remote file
 ftp.CloseFile();
 // notify completed upload
 Serial.println("File uploaded successfully");
 
 // wait
 delay(100);
}
void setup() {
 // begin serial communication
 Serial.begin(115200);
 // initialize micro sd
 esp_err_t errSD = init_sdcard();
 if (errSD != ESP_OK) {
 Serial.printf("Cannot initialize SD. Error code: 0x%x", errSD);
 return;
 }
 // begin wifi connection
 WiFi.begin(ssid, password);
 // wait for the connection to be estabilished
 while (WiFi.status() != WL_CONNECTED) {
 delay(500);
 Serial.print(".");
 }
 // notify successful connection
 Serial.println("");
 Serial.println("WiFi connected");
 // show the assigned IP
 Serial.print("IP: ");
 Serial.println(WiFi.localIP());
 // call the FTP upload procedure
 FTP_upload();
 
}
void loop() {
 // empty
}

How can I solve this? Thank you

• PMF: This line doesn't make sense to me: unsigned char * in PROGMEM = (unsigned char *) malloc(size); Either it's in PROGMEM, or it's on the heap. It can't be both. [I don't think that's your problem though, since PROGMEM doesn't exist on the ESP32 ASFAIK]
• timemage: Heh, that's all we need: progmalloc(). Or is it malloc_P() ? =) I haven't checked the ESP32, but usually non-AVR cores have dummy F(), PSTR(), PROGMEM, pgm_read_whatever, etc that just put things the only address space they have. So there's kind of a notion of having them in the sense that they don't cause errors for unrecognized identifiers.
• PMF: @timemage That's exactly what I think, yes. In the program above, it may not do anything, but it's just wrong. The data needs to be read to the normal RAM.
• timemage: @PMF, ah, that makes sense. I'd had a different reading of it before.
• user2959923: @PMF yes sorry, that was an error that I missed, indeed. Removing PROGMEM didn't change anything though. @timemage what do you mean? I successfully used malloc in another code, so I can confirm it gets recognized by the ESP32 core.

Answer 1

Solved, I changed the method a little.

In a loop, I read a portion of the file and write it to the opened FTP remote file, then I close both at the end of the cycle

// open FTP connection
ftp.OpenConnection();
// open local file
fileUpload = fopen("/sdcard/img.png", "r+");
// initialize file type
ftp.InitFile("Type I");
// change remote directory
ftp.ChangeWorkDir("/Shared/");
// define remote file name
String strName = "img.png";
// convert to char
const char *f_name = strName.c_str();
// create remote file with given name
ftp.NewFile( f_name );
// read all file in a loop
unsigned char buff[64];
while(!feof(fileUpload)){
 
 // read a portion of the file
 fread(buff, sizeof(char), sizeof(buff), fileUpload);
 // write that portion to the remote file
 ftp.WriteData(buff, sizeof(buff));
}
// close both the local and the remote file
fclose(fileUpload);
ftp.CloseFile();

• timemage: It would be good to get a more specific explanation of the cause. One thing you did was move away from malloc() which is a good idea. In the process it seems you fixed a memory leak that was caused by not having a corresponding call to free(). You also got rid of ftp.WriteData(in, sizeof(in));, which was writing only the first 4 four bytes of what in points to, because sizeof (in) is the sizeof of the pointer, not the size of the allocated block.
• timemage: For what remains, there's the possibility of fopen failing, which you should probably check before trying to use fileUpload. When it comes to feof and fread it's important to know that feof doesn't tell you when you're at the end of the file. It tells you when you tried to read past the end of the file. So to make the loop correct, you'd need to first attempt an fread, then check to see if it failed because of an EOF condition, and then either break the loop or write the data.
• user2959923: @timemage The reason for this solution is that the row fread(in, sizeof(unsigned char), size, fileUpload); tries to read size elements (size is the full size of the file) and tries to put those elements in the in char, but in char has only 4 bytes, regardless the malloc function.
• timemage: If you show up on freenode sometime, I'd go into what I'm thinking about that. For here, I'll just say I think you may need to keep looking at it a bit longer.