Updates include:

• Swapped out previous IMU for BNO-055 (easier to work with, more accurate)
• Switched from SPIFFS to LittleFS (for memory improvements)
• Added commands to increase debugging capabilities
  
• Removed Bluetooth as it was too heap inefficient. Will revisit later.

New Features:

• Added Thermal Infrared Sensor
• Added Color and Light Sensor
• Added an HTTP web Interface
  • Switched from Adafruit web server library to native ESP library (for security and function improvements)
  • Easier to eventually migrate to HTTPS
• File browser
• User Account System
  • Required authentication for web and serial interfaces

Updates include:

• Heap and Efficiency improvements across the board
  • Reducing heap churn and fragmentation by reducing the amount of string concatenation
    
    
• Sensors
  • Introduced a queue system for sensor initialization
  • Separate tasks are utilized for sensors
  • Thermal Sensor
    • Im trying to find ways to smooth out the output so it’s less jittery and flickery. The video below is extra bad because it’s at 2x playback speed. WIP
      
      
• Improved wireless connectivity options
  • There can now be multiple SSIDs saved to a device
  • SSIDs can be ranked by priority
    
    
• Web Interface
  • Visual overhaul
  • ‘Dynamic’ loading of sensor visuals and help commands depending on what sensors are connected
    
    
• General Security
  • WiFi Passwords and User Passwords are hashed, not viewable once entered.
  • Uses a basic hardware ID based hash.
    
    
• File System
  • In-browser file editing
  • File uploading to LittleFS storage
  • Folder and file creation
  • Marking certain folders immutable by user interaction

New Features:

• ESP NOW wireless communication
  • Remote Command Execution with output streamed back to the remote device (ssh like)
    
    
• ‘Automations’
  • The ability to queue commands based on time, event based triggers, simple ‘logic’, and on command.
  • Automations can be exported to a JSON format
  • Automations can be downloaded directly from Github (JSON only)
    
    
• Web Interface
  • Adoption of SSE
  • Ability to revoke sessions of users
    
    
• System Memory Allocation
  • Created function for memory allocation that replaces malloc where available with ps_alloc, which enables the initialization to use PSRAM if available. It will fall back to heap if PSRAM is disabled or it cannot allocate the space requested, or if the request is flagged to go straight to heap.
    
    
• Logging
  • Authentication attempt logging (always on)
    • Stored regardless of success or failure.
  • Sensor logging (optional)
    • Can be enabled by the user
  • Memory allocation logging (optional)
    • Works to provide a history of heap consumption
  • Command logging (always on)
    • Acts as a global log for both system commands and user executed commands

Updates Include:

• Migration from manual JSON parsing to the ArduinoJSON library
  • Increases ease of programming
  • Decreases code length
  • All JSON files stored in LittleFS utilize this
    
    
• ESP NOW wireless communication
  • A new ‘mesh’ mode exists, and allows messages to travel/hop between devices that are both paired and on the same encryption key to extend range
  • There are several kinds of messages that can traverse the network of devices
  • Through each hop the core message will remain the same while metadata builds on top of the original message. This provides a large benefit such as topology mapping, more intelligent routing
    
    
• Debugging
  • Added many more debugging flags to help provide more granularity to the programs output
  • Added many more debugging messages
    
    
• Sensor Settings
  • Added sensor settings to rotate the thermal sensor’s visual output, to help in configuring a device after its been built.
  • Removed group sensor buffer and replaced it with multiple smaller buffers to reduce race conditions and compute time with juggling another mutex separately for this.
    
    
• I2C Bus Management
  • Expanded upon the sensor queue to add all i2c devices into the mix
  • Added i2cMutex which allows multiple sensors to start and stop repeatedly without corrupting by preventing overcrowding of the I2C bus
  • ‘Dynamic’ Clock Adjustment
    • Allows for the sensors to request a clock speed change if necessary, without clamping back to a preset value.
    • Prevents unnecessary changes to only change clock when needed
    • Supports up to 8 different devices’s clock speeds between 100k and 800k
    • Gracefully errors out when encountering errors or early returns. No more crashing >_>

No video will be made available of this version.

I have already began a complete refactor to reorganize and modularize the codebase to enable feature selection at the time of compilation for maximum flash and iram savings, in addition to then having control of those selected during boot for heap/stack savings. The next update will leave behind the Arduino IDE in favor of the much more feature rich ESP IDF. From there I will work to de-arduino the code and make it as effective and efficient as possible before moving to my next goal.

New Features:

• Onboard Screen
  • Now featuring a 128×64 OLED monochrome display, the device finally has a way to display various different modes each with a different purpose.
    • Modes include: Sensor List, Sensor Data, System Status, Thermal Visual, Mesh Status, Custom Text, Logo, Animation, Boot Sensors, IMU Actions – These are named pretty literally. Currently it is only able to be changed via serial or web CLI commands.
      
      
• Game… Engine¿?
  • Rendering System
    • 2D View – Top-down rendering of everything
    • 3D View – Multi-Raycast rendering
  • Multiple control schemes
    • ‘Gyro Aiming’ + walk and attack buttons
    • Joystick aiming + walk and attack buttons
  • Attack System
    • Magic
      • Fire, Lightning, Ice and Magic Missile
      • Unique projectiles for each type of magic
    • Melee
      • Nothing yet
  • Leveling System
    • Rough XP system has been implemented. Does nothing.
  • Shop System
    • Nothing yet
  • Levels
    • ‘Random’ world generation
      • Seed based, and the seed is randomized
    • Multiple Levels/Biomes with Varying terrain height and types, as well as Columns, Walls, and Decorations to make it look less empty.
    • Varying enemy spawning
      • Not implemented yet
    • Based on seeds, and the seed will change when the page is refreshed.
    • Debugging
      • Rendering Debugging
      • Math Debugging
      • Time Debugging

Updates to make this project more enjoyable to use/work on:

• BuildConfig File
  • This is a file that is now used for configuration of the software at compilation time. Its used to enable or disable flags, which controls what software gets loaded into the ESP32.
  • This enables the user to flash different software on each device, saving both flash storage space, and enabling the usage of different features across multiple devices.
    
    
• OLED Screen menu revamp
  • This revamp focused on:
    • Adding Joystick and button support for user input
    • Making icons larger
    • Creating reusable elements for things like navigation text, scrolling menus, popup windows, notifications panels, an OLED keyboard, a file explorer, etc
    • Adding the navigation bar
    • Implementing nearly all the projects features
      • The OLED interface is now put behind a login screen
      • A CLI interface that shows what the serial interface would show
      • A Settings editor
      • A File Explorer
      • Paired Device configurator
      • Paired Device functions
      • Maps
      • Automation Viewer
      • Memory Footprint
      • Etc
      • The only missing elements are things that the OLED cannot display, such as the camera, as the monochrome screen can’t handle that resolution or the fact that camera has color.
  • This makes the OLED a viable screen for a low-cost, no-solder display option instead of an EYESPI screen.
    
    
• Project file reorganization
  • Project files now follow a general naming scheme:
    • Sensor related file names are named by what kind of sensor they are, then the ID of the sensor / chip, and then its display type.
      • Examples:
        • I2C Thermal Camera Sensor (MLX90640):
          • (Sensor functions file) i2csensor-mlx90640.cpp
          • (Sensor header file) i2csensor-mlx90640.h
          • (Sensor Interface file – OLED) i2csensor-mlx90640-oled.h
          • (Sensor Interface file – Web) i2csensor-mlx90640-web.h
    • System related files are named fairly literally.
      • Examples:
        • FileSystem
          • (File System functions file) System_FileSystem.cpp
          • (File System header file) System_FileSystem.h
        • DebugSystem
          • (Debug System functions file) System_DebugSystem.cpp
          • (Debug System header file) System_DebugSystem.h
            
            

On 2/24/26 I will change the Github to be public! 🎉

If you know me IRL and want to try this out let me know and I’ll send you free hardware 🙂

New Features/Updates:

• New Bonded Usage Mode:
  • Instead of just one ESP32 ‘per-device’ for this project, now there can be two ESP32s which share command registries, live statuses and more to work together
  • With this, you can have a device that can run all of the features in this project. Before the limiting factor was the instructional RAM, which limits the amount of libraries/features that can be included on one device.
  • Parses internal command manifest with fallback to bonded manifest if no matches are found
  • ‘Apps’ utilize either local or remote sensors
    • Maps and Games can use the GPS and IMU on a paired device if necessary
      • More functionality for other features coming shortly
    • Limitation: Camera and Microphone ‘Apps’ like Edge Impulse and ESP-SR require a greater data throughput than ESP-NOW is capable of. While they can be setup and the ‘Apps’ can be started, stopped, and the settings can be interacted with on the bonded device, they cannot be displayed live on bonded devices.
  • Leverages the ESP-NOW Communication system previously implemented
    
    
• Updated from V2 ESP-NOW communication to V3
  • Dropped the base64 encode in favor of binary due to less overhead
    • -30% overhead on every transmission
    • Faster chunked data sending
      • File Transfer
      • Large console output
  • Added new message types for paired devices
  • Enables the auth of cross-device commands with a session token created when devices are first successfully paired, and authenticated without directly needing user credentials.
    
    
• Notifications
  • Notifications can show
    • Connectivity changes of:
      • WiFi / Bluetooth / ESP-NOW / Paired Device
    • Manual command execution
      • Shows a notification for command execution
        • Shows different text/icons if command is executed on the local device or the paired device
        • Shows Success / Failure
          
          
• Added Offline Maps
  • Created Map Generator: Link
    • This allows the user to download an offline map of a specific area, which can be read by the device.
    • Having gone through many iterations, this features many customizable options to reduce feature count, feature complexity, feature name count, and more.
  • Added Web Interface and OLED maps interface.
    • The web interface is a full color version, nearly identical to what you see when you generate the map.
    • The OLED version is much less detailed, given it’s a 128×64 OLED screen. It shows a wireframe sort of view, skipping things like buildings and other features that would cause clutter. Despite this rendering change, its supporting functions work identically to the web interface map since they’re shared.
  • Added Waypoint files for each map
    • Stores:
      • Name of associated map file
      • Waypoint names
      • Map coordinates per waypoint
      • Associated files per waypoint
      • Notes per waypoint
        
        
• Added Camera + Microphone (Only for specific hardware)
  • Added Edge Impulse (Machine Vision)
    • I am getting decent results, but the models I train are generally low confidence – something I need to improve. I recommend a model with over 70% confidence to get results that aren’t jumpy.
    • Train models with Edge Impulse
    • Load the models onto the device and utilize the camera for object detection
  • Added ESP-SR Speech Recognition (Voice Control)
    • Utilizes the MultiNet 7 English Model
    • Commands are integrated into the per file command modules, allowing for automatic command list generation upon startup
    • The equivalent to “Hey Alexa” / “Hey Siri” for this is using the default “Hi ESP” – This can be changed later.
  • Photo capture and Video streaming
  • Microphone capture
    
    
• Added GPS I2C module
  • Added live location readings and map route tracking functionality
    
    
• Added IR Presence I2C module
  
  
• Added RTC I2C module
  • When added and configured, this can reduce boot times to 1/3rd of what they are otherwise.
    • I will be looking at the NTP function soon, it shouldn’t be that high of a impact.
      
      
• Added Bluetooth
  • Can act as a client and a server
  • 4 Max concurrent connections possible – reduced from 6 to save iram
    

Later releases will feature a ‘slim’ minimalistic case model, as well as a ‘recommended’ case model