You are viewing a static version of the site. Enable JavaScript to benefit from the
full functionality of this site and enable such things as forum posting.
SD Library not working for ESP32
ullix:
I want to use an SD card, and started your example Storage ->SD -> SD-Test. Library SD.h is included but won't compile. This message comes up repeatedly:
• Error at line 524 in file Sd2PinMap.h:
‣ #error Architecture or board not supported.
‣ SD [/home/ullix/Code/Arduino/libraries/SD]
◦ File.cpp
The package manager tells me the "SD Library" is installed as libsd-esp32, in version 1.1.7, and no update available. Looks like being the correct library?
So far I have used internal flash with both ffat and spiffs, and it worked.
ullix:
The exact same (I verified that) file does compile and upload on Arduino!
ullix:
I looked at the libs in uecide and arduino, and this is what I found:
uecide:
name=SD(esp32)
version=1.0.5
author=Arduino, SparkFun
maintainer=Arduino <info@arduino.cc>
sentence=Enables reading and writing on SD cards. For all Arduino boards.
paragraph=Once an SD memory card is connected to the SPI interfare of the Arduino board you are enabled to create files and read/write on them. You can also move through directories on the SD card.
category=Data Storage
url=http://www.arduino.cc/en/Reference/SD
architectures=esp32
arduino:
name=SD
version=1.2.4
author=Arduino, SparkFun
maintainer=Arduino <info@arduino.cc>
sentence=Enables reading and writing on SD cards.
paragraph=Once an SD memory card is connected to the SPI interface of the Arduino or Genuino board you can create files and read/write on them. You can also move through directories on the SD card.
category=Data Storage
url=http://www.arduino.cc/en/Reference/SD
architectures=*
Uijuijui! You are two point versions behind! How dare you?
But is that the cause of the problem?
Arduino claims for all architectures, and at least it does work with the ESP32.
matt:
The version UECIDE provides is the version that is bundled with the ESP32 core.
The version you are compiling is a copy of the Arduino library that doesn't work on ESP32.
You need to get rid of that library, or remove that library search path, so that the proper library will be selected instead.
BTW, architectures=* is meaningless. All that says is that the author thinks it will work with everything, but they don't know. There's too many architectures out there - too many cores. In this case it's a downright lie - since in the code it's plain that it only supports ARM, AVR and ARC chips - and then only a small subset of them. But, well, that's the total of Arduino's architectures, so that's "everything" as far as they are concerned - even though it obviously isn't.
ullix:
This highly confusing to me.
The "old" SD version 1.0.5 is indeed the one which works in both arduino and uecide.
When I loaded the example sketch into uecide, it did use the library not from inside uecide, but from some location outside. This is what my Preferences->Locations looks like:
Contributed /home/ullix/Code/uecide/libraries
libraries /home/ullix/Code/Arduino/libraries
So, with the uecide libraries on first place, why is uecide ignoring this, and going to a library found on second place?
Is there no way to bind a certain library to my sketch, apart from deleting the other library that I do not want?
matt:
At the moment, no, there is no way to select which library. In general if you provide a library in an externally specified location then that one is used in preference to the internal one. The highest priority location, though, is a folder called "libraries" inside your sketch folder . This allows you to keep a specific library version bound ("bundled with") a sketch.
You should not have an "SD" library in either of those locations though. UECIDE should find one in ~/.uecide and Arduino should find one in ~/.arduino15
However I am planning a way of manually selecting exactly which location a library is found in for a future release - however this requires a complete change in how libraries and their locations are stored internally in UECIDE.
matt:
Oh, and you say "old"... version numbers in libraries are meaningless. The SD library from Arduino hasn't actually really been updated in any significant way for many years. The "old" version from ESP32 is actually newer in that it has had support for the ESP32 added.
There's really no "old" and "new" - there's the Arduino SD library that works with Arduino boards. From that there is the ESP32 library that works with Arduino and ESP32 boards. Then there's the Teensy library that works with Arduino and Teensy boards. And then the chipKIT library that works with Arduino and PIC32 boards. And then the Launchpad library that works with Arduino and MSP430 boards. And then the ... etc ... etc ...
Each core maker took the libraries at the time and bent them to fit their boards. So there is no "old" and "new" - only "the library that is for this board family".
ullix:
Good Lord!
The highest priority location, though, is a folder called "libraries" inside your sketch folder. This allows you to keep a specific library version bound ("bundled with") a sketch.
does Arduino-IDE behave in the same way or is that uecide specific?
matt:
That's UECIDE specific.
ullix:
I think the idea is good and kind of obvious. The sigh is thatA-ide does not have it.
However5, it is always a loss when some solution works only in one place. Experience told that uecide is not that robust that it will always work, unless you are around to help out :-|.
matt:
UECIDE is pretty stable - you just seem to be cursed. I have users all over the globe, but you seem to be the only one at the moment having problems.
Hopefully all these problems you are having will help to make UECIDE better though. I have already started work on an all-new Library Manager system that allows multiple versions of the same library for the same platform to co-exist. The GUI is getting a complete rewrite from the ground up.
ullix:
I am using Storage libraries EEPROM, SD, FS, SPIFFS, and FFat. I noticed that all are availabel under .uecide except for FFat. Is there a reason for not including it?
matt:
Yes, it's a recent addition to the set of libraries that I hadn't seen, so didn't get added to the packaging files.
matt:
Just went to look at this and found that an error in the packaging meant it was building core version 1.0.2 instead 1.0.4 ... oops ... best do some testing of this core before I release it...
matt:
Ok... so I have done some initial testing and it looks good - so I've released it. Upgrading the ESP32 packages to 1.2.1 should give you ESP core 1.0.4. FFat is also available. Give it a test and see what happens.
ullix:
28 packages upgraded and all compiles and runs as before!
But FFat still isn't among the upgrades. I don't see it under .uecide ... Storage ...?
Was it supposed to be included? Where did it go?
matt:
It won't be there unless you actually install it. Upgrade only upgrades what is installed. It won't magically install something just because you happened to have a copy of it somewhere on your hard drive.
ullix:
Well, I used Plugin Manager, but without success, until I remembered that I need to refresh first, before I search.
So ffat is installed, but still not visible. Using my 'find' tool, I discovered that you have two sets of libraries, one under the hidden folder '.uecide/libraries' and one under 'uecide/data/libraries". The latter has the ffat.
Which of the 2 places does hold the standard installation, the hidden one or the other?
And how do I get it into my sketches, because they still long for Arduino-only?
Looking into my Preferences->Locations, I find this setting
Contributed /home/ullix/Code/uecide/libraries
libraries /home/ullix/Code/Arduino/libraries
where the first was left unchanged from the installation and does not even exist. The one where the last update went to is not even mentioned?
What should it read?
matt:
Everything from the plugin manager goes into ~/.uecide without exception. The "contributed" one is there for your convenience for manually adding extra libraries. If it doesn't exist then it doesn't care.
I guess the second one in the list you added to get at any Arduino libraries you manually installed.
All entries in that list are in addition to the system standard ~/.uecide/libraries location.
If you installed FFat through the plugin manager you should have:
$ find .uecide -name FFat.h
.uecide/libraries/Storage/esp32/FFat/src/FFat.h
ullix:
Everything from the plugin manager goes into ~/.uecide without exception.
This is definitively not true. See the picture. The FFat has a timestamp from yesterday, and apparently had been installed through this 28 package upgrade. It is in ~/Code/uecide/data/libraries. (upper pic)
The hidden folder ~/.uecide does not have any FFat! (lower pic)and all its files are from Oct 25.
ullix:
I was a bit to fast in proclaiming that the 28 upgrades worked as before; they don't!
I am now observing that every time I call the ESP web server via wifi, some watchdog triggers:
E (204511) task_wdt
: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (204511) task_wdt: - async_tcp (CPU 0/1)
E (204511) task_wdt: Tasks currently running:
E (204511) task_wdt: CPU 0: IDLE0
E (204511) task_wdt: CPU 1: async_tcp
E (204511) task_wdt: Aborting.
abort() was called at PC 0x401071cc on core 0
Backtrace: 0x4008d0a0:0x3ffbe160 0x4008d2d1:0x3ffbe180 0x401071cc:0x3ffbe1a0 0x40082441:0x3ffbe1c0 0x4016829f:0x3ffbc7a0 0x40108723:0x3ffbc7c0 0x4008ad89:0x3ffbc7e0 0x40089595:0x3ffbc800
The only thing I can see from this is the "async-TCP", which seems to point to the ESPAsyncWebServer?
I tested old backup-ed versions, which did work when I put them aside, and they also crash. So, likely it is due to the update.
Is there a way to roll back? Perhaps only the webserver?
EDIT: tested #include <WiFi.h>, which includes WiFiserver, and here WiFi and server do work. Not sure if that proves anything.
matt:
It could be that they changed the watchdog default settings between versions of the SDK. It sounds like something in your sketch in one of the callbacks is taking too long and blocking the code. You should always make sure that you yield() in any long loops. Or at the very least call esp_task_wdt_reset().
matt:
Have you installed FFat from the plugin manager? Tools -> Plugin manager -> Refresh -> Type "ffat" in the search box. Press the install button.
And delete your manually installed version.
ullix:
As I have said already: installed only by Plugin Manager, and no, zero, none, nüscht manually installed version! I did not even know there were libraries in the 'data' subdirectory until I discovered them with 'find'.
What is actually the intention of 'data' when all the sketches themselves should be the data to uecide?
Can I delete all libs under data and thereby get the equivalent of a rollback?
ullix:
There should not be any long loops that the async webserver would not take of itself. At least it did so in the past.
Since it already failed in the call to the root website, which is one of the shortest calls, I wouldn't even know where to insert any yields.
I think I need a solution to the easiest rollback, even if only for my own installation.
matt:
The "data" directory (~/.uecide by default) is where UECIDE stores everything that it owns. That is, anything installed from the plugin manager, any configuration files, any packages, etc.
Unless you have changed that in preferences, which you really shouldn't unless you have a very good reason, then all packages will be installed there. That is 100% guaranteed. I know, because I programmed it to do that.
The .uecide folder is the equivalent to Arduino's .arduino15 folder (and previous to that the .arduino folder).
Sketches are not "data" to UECIDE. They are sketches. Data to UECIDE is things UECIDE needs to run and perform its job. Sketches are not that. They are things you have written to use in UECIDE. UECIDE can run without sketches, but it can't run without its data.
If you don't have a manually installed version of FFat.h and only a Plugin Manager installed version, yet your only FFat.h file is in /home/ullix/Code/uecide/libraries then I would suggest that you have messed up your installation by changing your UECIDE data directory in Preferences to a non-standard place (/home/ullix/Code/uecide) and you're confusing the whole system.
Under normal circumstances you have no reason to access UECIDE's data directory, so no reason to move it somewhere else where it's more visible. That functionality is really intended for running parallel installations or portable installations with shared data (for schools on a network for instance) and takes care to set it up right.
matt:
Every package you have ever installed is saved in a cache. You can manually reinstall any old version of a package from that cache.
Open plugin manager
Select "File" -> "Install package"
Navigate to /home/ullix/.uecide/apt/cache
Select the package you want to install
You will most likely need to type the path in manually since the Open File dialog won't be able to see the .uecide folder.
You could also copy any of those files out to somewhere else to save them for later if you want.
ullix:
Your 'data' folder seems to have the very same info that the hidden-uecide folder has, except that the latter also has the file preferences.txt.
But when I rename either of these folders, uecide recreates them, yet seems to want to have both folders with redundant info present?
Why then must both be present? How must I rename the data folder in the Preferences in order to not have two folders with the same content, which, however, may change in only one after an upgrade???
matt:
They mustn't. Are you sure you didn't change the data location in your preferences?
In Preferences -> Locations the "Data location" should be left blank unless you have a very good reason for changing it.
If you want to change the default location for saving sketches that's the "Sketchbook location" on the same page.
ullix:
Yes, I did change the data location. Without a manual to explain, my interpretation of what it meant was obviously different from what you programmed ...
Now the data location is back to blank; I all 'data' in the filesystem and also deleted the hidden '.uecide' and rebuilt. And now all seems to be compiling and running well.
The whole files system mess started with me wondering why FFat was not among the uecide offerings. Now it is, and is even installed, as Plugin manager tells me. But my scripts are still reaching over to Arduino to load FFat. Why?
Upon closer review I noticed that in addition to FFat, uecide is also ignoring its own installed libraries for NTPClient, and PubSubClient.
What determines the priority of uecide loading the libraries? Obviously not their availability within the uecide framework. And how can I force the 'own' libraries?
matt:
The basic order of priority, lowest to highest, is:
UECIDE internal (plugin manager)
Contributed / manually added libraries
Libraries in the sketch folder
It is done that way so that you have the choice to override the internal UECIDE ones with newer versions if you want. If you want to use the internal UECIDE ones then don't have them in your contributed area.
If you want to share libraries between Arduino and UECIDE on a per-library selective basis I would suggest symbolic links to connect them between your Arduino libraries folder and a UECIDE libraries folder.
ln -s /home/ullix/Code/Arduino/libraries/SomeLibrary /home/ullix/Code/UECIDE/libraries/
That will give the same library in both locations, and you can do it selectively for each library you want to have available on both platforms.
However when I have finished the rewrite of library handling you will have:
Priorities assigned to each location with the internal libraries being a "mid" priority
The ability to pick and choose exactly which copy of a library to use for any given sketch
The ability to point a library in a sketch to a specific location that isn't even known about by UECIDE.
ullix:
This is a very helpful information, thanks. Wishing you a Happy New ... Rewrite!
As I am also becoming more sensitive to the libraries, I choose, could you please explain, what makes you include any library?
matt:
> As I am also becoming more sensitive to the libraries, I choose, could you please explain, what makes you include any library?
How do you mean?
ullix:
My concern is licensing, support, dependencies, things which may be troublesome in the future.
But that should be independent of my question how you make the selcetion of libraries which are offered via uecide?
matt:
In general you already know what library you want. The libraries in UECIDE are generally either the ones bundled with the cores or popular ones directly from Github. If you are undecided between different libraries that's not really to do with UECIDE - if one of the libraries (or some of the libraries) happen to be in UECIDE then you have a convenient way of installing them - otherwise just download them and put them in the contributed folder - as you would with the Arduino IDE.
How I choose which library to use? Well, that's not really going to be that helpful, since I mostly work with chipKIT, and there's not as much direct library support for that, so I find myself writing my own libraries from the ground up more often than not.
ullix:
There seems to be a problem with the libraries priorities.
I am calling <SSD1306.h> (OLED display), which exists in multiple libraries; one is installed via plugin manager (Package name: libdisplaycore-driver-ssd1306, Installed: 2.1.0). But my code presently needs a different one (ESP8266_and_ESP32_Oled_Driver_for_SSD1306_display).
So I put the latter into the 'libraries' folder in the sketch's folder, which - as you stated - has highest priority.
This library is also under a contributed library, that should have 2nd priority.
Yet, the sketch still selects the uecide library, which supposedly has lowest priority? What is wrong here?
matt:
Actually the order might be different. Looking at the code (I wrote this part a long long time ago...) the loader routine first loads the libraries from the "extra" (contributed, etc) locations, then the sketchbook, then the internal libraries. It really shouldn't be that way around, and I don't know why I did it that way.
The library loading is a bit of a mess at the moment, and it will be so good once I have finished the overhaul and got it working properly. All I can suggest right now is uninstall the DisplayCore one (which is intended really for chipKIT boards, though I have used it with ESP32 as well) from the plugin manager.
ullix:
the future will be marvelous ;-)
I have some trouble code on the I2C bus fighting against each other, but can't pinpoint the problem. One code is for a BME280 sensor, the other for the OLED display. So I replaced my old code (ESP8266_and_ESP32_Oled_Driver_for_SSD1306_display) with that from uecide. I noticed that you/your company has the copyright on the latter, so I thought I might ask a question:
I got the OLED working a few moments ago with your stuff. First thing I noticed is that it is MUCH slower than the old library, like about 10x slower! That seems to be a huge difference. Is there a reason for this?
My program still crashes as soon as the BME280 is activated, so this is not the solution anyway, but the sluggishness would have made it a real problem. Just updating the few numbers on the OLED (128x64, b&W) takes 900...1100 ms.
matt:
It's because my library is a full graphical one with buffering. That means that it's got more data to send (an entire frame IIRC) at once. It's not really any good over I2C. Better over SPI.
Actually I have "dirty" status on segments of the buffer, so it only updates the right bits - but that is row-based at the moment, so a whole row has to be transferred at a time.
I don't do anything with the clock speed, so increasing the Wire clock speed may improve performance. However, the ESP32 I2C is bit-banged, so I don't know how much faster that would go...
