Acoptex.com

members

Basics: Project 084b

Project name: ESP32-CAM Development Board with camera - Using ESP-IDF and ESP-WHO

Tags: ESP-IDF, Ai-Thinker, ESP32-CAM, Development Board with camera, OV2640, OV7670, seeedstudio, FTDI programmer, ESP-WHO

Attachments: windows_install_prerequisites.sh

In this project, you needed these parts (Dear visitors. You can support our project buy clicking on the links of parts and buying them or donate us to keep this website alive. Thank you):

1. ESP32-CAM Development Board with camera 1 set

2. Arduino IDE ( you can download it from here  )

3. USB to TTL/Serial adaptor/converter 1 pc

4. Jumper cables F-F, F-M

5. Micro SD card and SD card adapter 1 pc

6. WiFi Antenna, Compact Internal WiFi Antenna with U.FL connector and 5.9 inch cable 1 pc (optional)

General

We will learn how to setup the Ai-Thinker ESP32-CAM Development Board with Espressif's ESP32 development environment and the ESP-WHO libraries for face detection and recognition.

What is ESP-WHO?

ESP-WHO is a face detection and recognition platform that is currently based on Espressif Systems' ESP32 chip.

ESP-WHO supports development of face detection and recognition applications based around Espressif Systems' ESP32 chip in the most convenient way. With ESP-WHO, you can easily build up face detection- and recognition-featured applications, for instance:

• A coffee machine that brews coffee according to your taste preference;
• Home applicance that will shut off the electricity automatically when unsupervised children are operating them;
• And other more applications that suit your needs.

In general, the ESP-WHO features will be supported as shown below.

In ESP-WHO, Detection, Recognition and Image Utility are at the core of the platform.

• Image Utility offers fundamental image processing APIs.
• Detection takes images as input and give the position of face if there is a face. It is implemented with MTMN model, which refers to MTCNN and MobileNets.
• Recognition is to identify the particular person, and it needs the results of detection. It is implemented with MobileFace model.
• Optimization is mainly to increase the precision of the inference, and to accelerate the whole process. But also it might change the structure of the network, update the coefficients, refactor the code, etc.

Both input and output are flexible.

• Image sources could be input via camera. However, we don't provide many drivers right now, those for other camera modules will be released in the future.
• Results could be output and displayed through Command line, LCD or even website via Wi-Fi http service.

To run ESP-WHO, you need to have a development board which integrates a ESP32 module that has sufficient GPIO pins and more than 4 MB external SPI RAM. Either ESP-WROVER-KIT or ESP-EYE can be a good choice as the test board. The recommended resolution of input image is QVGA (320x240). As for choosing camera as an image offer, make sure that the ESP32 module you choose offers specific pins that your camera needs. This library supports OV2640 and OV3660 cameras only.

ESP32 camera boards

There were some ESP32 camera boards launched previously such as ESP32-PICO-tinyCAM, but most seem to have been phased out. Seeed Studio had AI Thinker ESP32-CAM development board with a 2MP camera for $9.90. This development board can be programmed with ESP-IDF as explained in details in Github or in Arduino IDE. The demo code takes a QR code, and outputs it as ASCII code in the serial console. Another option - M5Stack ESP32 camera (ESP32CAM) board sold for $13.90 shipped on Banggood. The latter features the same OV2640 camera module, does not come with a micro SD card, but includes a USB port, and a grove connector, as well as pads for soldering MPU6050 & BME280 sensors. Another advantage is support LiPo battery via IP5306 charger chip.

We will learn in this project about the ESP32-CAM Development Board with camera (for more details please check seeedstudio.com website), how to connect it with USB to TTL converter or Arduino Uno to your PC/laptop, how to update firmware and how to get AT  communication.

Understanding the USB to TTL converter

You can read more about them here.

Understanding the ESP32-CAM Development Board with camera

The ESP32-CAM has a very competitive small-size camera module that can operate independently as a minimum system with a footprint of only 27*40.5*4.5mm and a deep sleep current of up to 6mA.

ESP-32CAM can be widely used in various IoT applications. It is suitable for home smart devices, industrial wireless control, wireless monitoring, QR wireless identification, wireless positioning system signals and other IoT applications. It is an ideal solution for IoT applications.

ESP-32CAM adopts DIP package and can be directly inserted into the backplane to realize rapid production of products, providing customers with high-reliability connection mode, which is convenient for application in various IoT hardware terminals.

The ESP32-CAM Development Board with camera doesn’t come with a USB connector, so you need an USB to TTL/Serial adaptor/converter to upload code through the  serial pins (U0R and U0T pins).

Features:

• The smallest 802.11b/g/n Wi-Fi BT SoC Module
• Low power 32-bit CPU,can also serve the application processor
• Up to 160MHz clock speed，Summary computing power up to 600 DMIPS
• Built-in 520 KB SRAM, external 4MPSRAM
• Supports UART/SPI/I2C/PWM/ADC/DAC
• Support OV2640 and OV7670 cameras,Built-in Flash lamp.
• Support image WiFI upload
• Support TF card
• Supports multiple sleep modes.
• Embedded Lwip and FreeRTOS
• Supports STA/AP/STA+AP operation mode
• Support Smart Config/AirKiss technology
• Support for serial port local and remote firmware upgrades (FOTA)

Specifications:

• Dimensions 40.5mm x27mm x4.5mm
• Weight G.W 20g
• Battery Exclude
• Package DIP-16
• SPI Flash Default 32Mbit
• RAM 520KB SRAM +4M PSRAM
• Bluetooth Bluetooth 4.2 BR/EDR and BLE standards
• Wi-Fi 802.11 b/g/n/
• Support interface UART,SPI,I2C,PWM
• Support TF card Maximum support 4G
• IO port 9
• UART Baudrate Default 115200 bps
• Image Output Format JPEG( OV2640 support only ), BMP,GRAYSCALE
• Spectrum Range 2412 ~2484MHz
• Antenna Onboard PCB antenna, gain 2dBi
• Transmit Power 802.11b: 17±2 dBm (@11Mbps); 802.11g: 14±2 dBm (@54Mbps); 802.11n: 13±2 dBm (@MCS7)
• Receiving Sensitivity CCK, 1 Mbps : -90dBm; CCK, 11 Mbps: -85dBm; 6 Mbps (1/2 BPSK): -88dBm; 54 Mbps (3/4 64-QAM): -70dBm; MCS7 (65 Mbps, 72.2 Mbps): -67dBm
• Power Dissipation Turn off the flash lamp:180mA@5V; Turn on the  flash lamp and turn on the brightness to the maximum:310mA@5V; Deep-sleep: Minimum power consumption can be achieved 6mA@5V; Moderm-sleep: Minimum up to 20mA@5V; Light-sleep: Minimum up to 6.7mA@5V
• Security WPA/WPA2/WPA2-Enterprise/WPS
• Power Supply Range 5V
• Operating Temperature -20 ℃ ~ 85 ℃
• Storage Environment -40 ℃ ~ 90 ℃ , < 90%RH

In the package you will get 1 x ESP32-CAM Development Board, 1x camera Module-OV2640-FPC-24P-0.5mm-21mm- 2 Megapixel and 2x headers-2.54mm-1x8pin.

This product has been discontinued at Seeed Studio.  They highly recommend to try & choose Sipeed series products: Starts from $7.9,The 1st RISC-V 64 AI modules/boards for Edge computing, machine vision, speech recognition.

Make AI embeded to any IoT device possible! The new released Maixduino was designed in an Arduino Uno form factor, with a ESP32 module on board together with the MAIX AI module. The support of arduino compatible interface as well as Arduino IDE enable you update your arduino projects from IoT to AIoT easily without any additional cost! The maixduino kit provides an OV2640 camera module and a 2.4 inch TFT LCD for quick start machine vision projects, and onboard MEMS microphone for experiencing speech recognition promptly.

On-Board vs IPEX Antenna

ESP32-CAM Development Board with camera can be with the antenna jumper set for an external antenna or for the on-board antenna. You can see the two different settings below:

If you have a low frame rate or poor reception this might be the problem. You can either move the jumper with a bit of very careful soldering or buy an IPEX WiFi antenna which will improve the signal.

Applications:

• We can easily build a simple camera with this board and save the photos in an SD card using the embedded SD card slot the board offers.
• We can then modify this project to detect humans and save an image when one is detected.
• We can recognize human faces and act accordingly.
• We can build a surveillance camera with the ESP32 Cam board.
• We can also stream video using this board.

You can find the ESP32-CAM schematic here, ESP32-CAM development board specification is here, camera specification is here.

Signals and connections of the ESP32-CAM Development Board with camera

ESP32-CAM pinout:

GND - ground pin. Connected to Arduino board GND pin.

5V - external power supply pin 5VDC 2A

VCC - power supply pin. Connected to 5V pin.

3V3 - power supply pin. Connected to 3V3 pin.

U0TXD, U0RXD - Serial pins. You need these pins to upload code to your board.

GPIO 0 - pin which determines whether the ESP32 is in flashing mode or not. When GPIO 0 connected to GND, the ESP32 is in flashing mode.

The following pins are internally connected to the Micro SD card reader:

• GPIO 14: CLK
• GPIO 15: CMD
• GPIO 2: Data 0
• GPIO 4: Data 1 (also connected to the on-board LED)
• GPIO 12: Data 2
• GPIO 13: Data 3

Wiring

ESP32-CAM module    FTDI programmer

U0TXD (GPIO 1)         RXI                 

U0RXD (GPIO 3)         TX0 

GND                          GND

5V                             5V

GND and GPIO0 connected just for uploading the program                                

Step by Step instruction

We are using PC with Windows 7 64 bit OS. Very important: please note that username must be without space (single word only). For example we have AlexBestman.

1. Adding ESP32 Development Environment

1. Go to msys2.org website and download the 32bit version (msys2-i686-20190524.exe) of MSYS2. Please note that MSYS2 can't be installed on FAT* partitions and current MSYS2 can't be installed on Windows XP anymore.
2. 
3. Open msys2-i686-20190524.exe. The installation process starts.
4. 
5. Click on Next button.
6. 
7. Specify the directory where the program will be installed and click on Next button.
8. 
9. Click on Next button.
10. 
11. Wait till the installation process completed.
12. 
13. Uncheck Run MSYS2 32-bit now checkbox and click on Finish button.
14. Go to C:msys folder and double-click mingw32.exe file to run the application.
15. Download and copy the  windows_install_prerequisites.sh file to C:msys folder.
16. 
17. Copy or paste the command C:/msys32/windows_install_prerequisites.sh and press Enter button on keyboard:
18. 
19. You can get the error message similar to this: ***fatal error - cygheap base mismatch detected. Close the MSYS2 terminal window and again navigate to C:msys and double-click mingw32.exe to re-open the application. Press the up arrow on keyboard to see the C:/msys/windows_install_prerequisites.sh command and press Enter button on keyboard.
20. 
21. When the script has completed, type or paste the following commands one at a time:
22. cd ~
23. mkdir esp
24. cd ~/esp
25. git clone --recursive https://github.com/espressif/esp-idf.git
26. 
27. If you run into an error: cannot fork() type or paste the command below: c:/msys/autorebase.bat and run the git clone command again.
28. All the APIs and libraries for the ESP32 have been installed in c:msyshomeAlexBestmanesp on our PC.
29. 
30. A user profile script needs to be added so projects will build without the path to these files having to be entered every time. Find the path to the ESP-IDF. We have C:msyshomeAlexBestmanespesp-idf . 
31. Create a new file named export_idf_path.sh in C:/msys/etc/profile.d/ folder.
32. Open it in Notepad (we used Notepad++) and add the following line to the new export_idf_path.sh file replacing the IDF_PATH with your own. Check you are using forward slashes: export IDF_PATH="C:/msys/home/AlexBestman/esp/esp-idf". Save and close this new file.
33. Close the MSYS2 terminal window and then double- click mingw32.exe to open it again. Check if IDF_PATH is set by typing the command: printenv IDF_PATH .
34. 
35. To complete set up, paste the following commands in the MSYS2 terminal window:
36. cd ~
37. C:/msys/mingw32/bin/python.exe -m pip install --user -r C:/msys/home/AlexBestman/esp/esp-idf/requirements.txt
38. The ESP32 development environment is now set up.

2. Installing ESP-WHO Face Detection and Recognition Libraries

ESP-WHO Face Detection and Recognition Libraries

1. Type or paste the following commands in the MSYS2 terminal window:
2. cd ~
3. git clone --recursive https://github.com/espressif/esp-who.git
4. 
5. If you look in c:msyshomeAlexBestmanesp-who folder you will see the ESP-WHO libraries and examples have been installed.
6. 

3. Flashing the code to the ESP32-CAM Development Board with camera using FTDI programmer

1. Do wiring. Very important: GPIO 0 (IO 0) must be connected to GND otherwise you will not be able to upload code.
2. Attach WiFi antenna (optional). It will help you to increase WiFi range.
3. Attach OV2640 camera.
4. 
5. 
6. Plug your FTDI programmer into your PC USB port.
7. Open Windows OS Device Manager and make a note of the port number of the FTDI programmer (We have COM10).
8. 
9. In the MSYS2 terminal window (if you closed it go to C:msys folder and double-click mingw32.exe file to run the application) 
   type or paste the following:
   cd ~/esp-who/examples/single_chip/camera_web_server/
   make menuconfig
   type or paste the following:
10. cd ~/esp-who/examples/single_chip/camera_web_server/
11. make menuconfig 
12. 
13. If some Python requirements are not satisfied you will need to install them additionally.
14. Type these commands to install them:
15. pacman -S mingw-w64-i686-python2-pyserial
16. pacman -S mingw-w64-i686-python2-future
17. pip install click
18. pip install pyelftools
19. pip install "pyparsing>=2.0.3,<2.4.0"
20. pip install "cryptography>=2.1.4"
21. You will be able to open the menu window after installing all these additional packages.
22. 
    
23. In the menu window that opens you need to change the following settings: Serial flasher config -> Default serial port (Change to the port number shown in Windows OS Device Manager – ie COM10); Camera Web Server -> WiFi Settings -> (Add your WiFi SSID and Password); Camera Web Server -> Camera Pins -> Select Camera Pinout -> (Select ESP32-CAM by AI-Thinker); Component config -> ESP32-specific -> SPI Ram config -> Type of SPI RAM chip (Select Auto-detect). Save the configuration.
24. 
25. 
26. 
27. 
28. 
29. 
30. 
31. 
32. 
33. 
34. 
35. Press the ESP32-CAM Development Board on-board RST (RESET) button.
36. In the MSYS2 terminal window type the command: make flash
37. If you get an error that the device fails to connect. Check that the IO0 pin is connected to GRD and press the ESP32-CAM Development Board on-board RST (RESET) button and try again.
38. When the device has completed flashing. Unplug IO0 from GND and type the command in the MSYS2 terminal window: make monitor
39. Press the ESP32-CAM Development Board on-board RST (RESET) button and watch the start up sequence in the terminal.
40. Look for the IP address the ESP32-CAM Development Board has been given on your network camera wifi where it outputs got ip: xxx.xxx.xxx.xxx (We have 192.168.1.110).
41. You can access your camera streaming server on your local network. Open any internet browser (IE,Google Chrome,..) and type the ESP32-CAM Development board IP address (we have http://192.168.1.110). 
42. You should be able to see a GUI on the left where you can control elements of the camera, set face detection and face recognition. 
43. Click Get Still to take a photo. 
44. Click Start Stream to see a video stream from the camera.
45. 
46. You can do face recognition and detection. First, you need to enroll a new face. It will make several attempts to save the face. After enrolling a new user, it should detect the face later on (subject 0).
47. That’s it. Now you have your video streaming web server up and running with face detection and recognition.
48. 

Summary

We have learnt how to setup the Ai-Thinker ESP32-CAM Development Board with Espressif's ESP32 development environment and the ESP-WHO libraries for face detection and recognition.

Libraries:

• ESP-WHO library. You can read more here.

Project resources:

• See attachments on the begining of this project description