Introduction: Snore-O-Meter: Using AI to Detect Snores

"The advantage of the snorer is that he sleeps well"... That's what I tried to explain to my wife when she told me that I'm snoring... Information technology didn't make her lough.

So I decided to gather some facts to help ME understand the problem, and mayhap discover a solution...

Any interesting facts about snoring

  • 25% of citizenry saw wood on a regular foundation, and almost 50% of adults will snore at some point in their lives.
  • Snoring stool happen during any stage of sleep.
  • Snoring occurs when your soft palate and uvula hover while you'ray inhalation (get wind picture).
  • Snoring is hereditary: 70% of snorers give a familial link.

And finally:

Snore is the fractional stellar cause of divorce in the US. Fortunately, I don't live in the The States...

How disturbing is it?

Whereas the average volume of a saw logs is around 38 dB, scientific studies have classified snoring sounds as mild-mannered (40-50 dB), moderate (50-60 dubnium), Beaver State grave (> 60 dB). Others accept measured snoring sounds adequate to 90 dubnium. The dB sound scale of measurement is a logarithmic scale, meaning that adding 3 dB is equivalent weight to multiplying the sound's amplitude past 2: releas from 60 to 90 dB is the same as multiplying the sound intensity by 1000.

Ultramodern dishwashers are around 62 hahnium, and a hand blower or a thermic lawnlower can reach functioning to 90 dB. The Boeing 777 was also measured at this horizontal at take-off...

It is said that the loudest stertor ever recorded was 120 decibels, the sound of a jackhammer! The dB scale says that it can cost deadly after 15 seconds. The Guiness Book of Records only relates a 93 dB sound record.

Proudly introducing the Saw wood-O-Meter

Imagine you sleep next to a hair dryer! Or imagine Maine quiescency under an incus... I had to set something: how could Arduino help me?

People are more likely to stertor when sleeping on their second. With gravity, the tongue and woolly palate more easily fall to the back of the throat which partially closes the airway. To prevent snoring, it's best to sleep on the side.

This is a first idea: how to make me log Z's on the side? Maybe some device to force ME to move on the side when I saw logs? But when do I snore?

This was then that I opinion all but a snore sound demodulator : the Snore O Meter (c) ! Easier said than done... Find below.

The 2 funny pictures are from a French comic book, edited past Glénat.

Step 1: What Do You Call for ?

Obviously, this will cost dealing with real clock sound analysis. An Arduino board, much as a Nano OR Uno is a bit bit as well small for this tax. This requires many remembering and higher computation speed. And so I distinct to habit an ESP32, which I'm acquainted (see my other Instructables). It is a dual Congress of Racial Equality running at 240 MHz, BT + WiFi equipped System on Chip (SoC), with 4 MB ostentation memory.

Flier of material:

  • ESP32 board,
  • 128 x 64 I2C OLED display (make a point it's I2C, entirely 4 pins: VCC, GND, SDA, SCL),
  • a push button,
  • a MAX446 mike with adjustable advance,
  • one small breadboard, some wires.

The overall cost is under 15 USD.

The physical phenomenon diagram will remain the same throughout this Instructable:

  • The ESP32 module powers all the components, using the 3V3 pin,
  • Connect the OLED to I2C pins: SDA to GPIO21, and SCL to GPIO22
  • Tie the analog input signal of the microphone to the GPIO35
  • Connect the push betwixt GPIO19 and GND

That's all folks!

Sound analysis is done using Fourier transform. There is a smashing library available for Arduino and ESP32 boards, made away Enrique Condes. Merely I wasn't satisfied by it, as it seems to comprise modified at full frequencies. And then I decided to look for another one, and adapt information technology to the ESP32.

Stone's throw 2: First Trials

First try

My very first estimation was to psychoanalyse the snore sounds. The main problem being that I'm usually drowsing when I snore, which prevents me from using the ESP32...

So I first-class honours degree decided to essa on misrepresent snoring sounds, that I could produce while awaken. I thus wrote a program for analyzing the frequence content of the sounds, glorious by this project, written for an ESP8266. I made it with the Lapp exhibit, push button and mike, but programmed it along a Wemos D1 Mini dining table.

I bequeath present this project in more details in another Instructables subsequently (I improved IT quite a lot since). You privy see on the video that Ludwig Ludwig van Beethoven ( ^-^ thanks! ^-^ ) gave me a hand on it... Think of that 2022 is the 250th anniversary of his birth.

This first run wasn't very conclusive: I couldn't find any obvious blueprint in the frequence spectrum that would be several rather signature of a snoring sound.

However, the Journal of Catch some Z's Research has published a paper called "How to measure eupneic? A comparison of the microphone, cannula and piezoelectric detector" in 2022 (ref. J Sleep out Res. (2016) 25, 158–168) that indicates that snoring sounds of people from Iceland have a fundamental frequency range mainly in the 70 - 160 Hz range.

This was encouraging: I am non the only one who addresses this trouble...

Second trial

So I decided to use more appropriate tools.

Audacity is a free software which can record and analyze sounds. I installed it on a laptop, that would viewpoint aside var. my have intercourse during the night. I set it adequate record every sounds in the bedchamber and save that in a wav arrange.

Audacity can past compute a spectrograph, which displays the sound spectrum as a function of time. The following picture shows one of the recorded spectrograms, centered along whatsoever snoring events that I spotted (during the night 22-23 March - what wouldn't we do to keep ourselves busy during lockdown?):

The and so-called snore events are the regularly spaced noble (rectangu-lish) floater on the speed part of the plot. Hera is a zoom on this part of the image:

The menstruation of the snores is around 5 seconds, which must atomic number 4 the duration of a breath when I am in deep slumber.

The respectable news was that it seems possible to discover a frequency touch of a stertor... the bad news was that information technology is fundamentally unlike from the results of the paper cited above: the frequency drift of interest is betwixt 4000 and 5000 Hz! Other replaceable measurements I did showed the unchanged rather result: I have to hunt for snores in this frequency range. Maybe that's because I don't live in Iceland ???

Artificial intelligence service?

But writing a cypher that must ut real time Fourier transform and dissect the frequency content over running periods of 5 seconds, piece not acquiring confused with other sounds that also happen at night (chiefly body movements or claps of the clapper) and sometimes part similar frequencies (understand the vertical red ink line at 48:10?), seemed a little bit complex to me. You said it can I be sure that this frequency range would be the same for someone else?

Soh I decided to grow my tending to the hot topic at hand: Bradypus tridactylus. Artificial Intelligence! Everyone's talking virtually it, everyone's doing it, why wouldn't I?...

Step 3: How a Neural net Whole kit

So I figured, let's move on. The buzz-word of the moment: Artificial Intelligence! It's exactly the right-hand tool for this kind of need. An Artificial intelligence course of study learns to recognize a pattern, or better yet, a family of replaceable patterns. Once the learning phase is done, a second and much simpler syllabu tries to discover the things it learned among the information information technology receives.

A neural network tries to reproduce the way our brain learns. In squabby, a NN is made of single layers of sol-called 'neurons', which process a small part of the information it receives. The processing results of the first layer is transmitted to the second unrivalled for another processing form, and incrementally, the network identifies patterns or similarities among the data IT was fed with. Around talking, the network is a kindly of very big and involved interpolation machine: it uses input data to bump the scoop function which prat ready unknown data into its master dataset.

There are many specialized sites and tutorials dealing with Neural Networks on the WWW. Simply genuinely not many examples of this sort of application on the ESP32. Notwithstandin, the ESP32 is very good suited for this: it is quick and has lots of RAM.

Along a PC, the near fashionable AI framework today is Google's Tensorflow, which can be easily implemented using the library Keras. Keras provides a high level library of functions that enable a seamless implementation of Tensorflow and fast experiment of neural networks. Information technology is written in Python, likewise as TensorFlow Nonfat... not suitable for ME, as I wanted to code in C on the ESP32.

Only a few developments exist happening ESP32 Cam for facial recognition, victimisation micro-python, but that's about it.

For the sound analyzer I couldn't find an FFT library I liked, so I decided to adapt other one, written in C, which I found connected the Internet. I might too suffice the same for the AI! So I searched for AI libraries and codes in C, with little or no dependency, and if possible non too heavier-than-air. Hera are a few of them:

  • KANN
  • Little-ANN
  • Multi-Bed Perceptron Library

At that place are certainly others. I haven't tested them all, of run. Finally, I opted for TINN, by Glouw.

TINN (Tiny System Network) is a 200 delineate dependency free neural net library written in C99.

I chose information technology because it's lightweight, has no dependency, the code is easy to understand, so it's prosperous to adapt and the porting was hot. Just i Oregon two functions to accommodate (mainly the random number generator - only this is true for all NN libraries, and functions for reading and writing files).

Step 4: Create the Dataset

One drawback of neural networks, is that they motivation a lot of input file to workplace and learn accurately.

First-class honours degree step: criminal record snore sounds

The NN must rehearse on the input data, to create its internal image of the data organization and extract or identify the underlying relationships. So I had to make over this input dataset.

A recording program was made connected the basis of the vocalize analyzer code (see Step 2), I just needed to make a nice user interface and add the possibility of redeeming the recorded information in a file. The ESP32 has an internal file away system, called SPIFFS, described here, with the related to API and a powerful depository library, which enables to make over, read, write, and cancel files. These files are stored in the SPI memory, which retains its content when the ESP32 is non supplied.

IT is also possible to upload files from your estimator to the SPIFFS of your ESP32, for object lesson via the Arduino IDE aside using this add-on. Designed by me-nary-dev, a great Bulgarian developper connected ESP8266 and ESP32, it can simply constitute used as follows:

  1. Create a folder called Data in the current forlder of your ESP32 sketch,
  2. Put in that folder the files you want to upload in the SPIFFS
  3. Enveloping the serial monitor (very important, otherwise the add-on will non work),
  4. In Arduino IDE: Tools > ESP32 Sketch data uploader
  5. If necessary, push the button(s) on your ESP32 module, just as you do for uploading a sketch.

Unfortunately, there is no easy room of extracting a file from the ESP32 SPIFFS. So I just wrote a easy sketch that copies in the Serial Monitor the content of all the files in the SPIFFS (they must be ASCII encoded, not binary files). I just have to select the parts I want to keep in the monitor, and re-create / paste them in a file using a file editor (Notepad++ is great for that).

This sketch is called Dump_SPIFFS.ino (view below). IT's useful if you want to save your dataset on your computer once you give birth created it.

How to record yourself?

Create a folder named Acquisition_ESP32, and put the following files in it: Acquisition_ESP32.ino, functions.h, params.h

Load the adumbrate called Acquisition_ESP32.ino on your ESP32 and allow it run. The sketch showtime searches for an already existing file in in the ESP32's SPIFFS. If you cause not want to keep the file, just push the button to erase IT, otherwise, the taped information will be appended to it (you have 3 seconds to push the button).

Then the sketch runs iteration acquisition phases. To start such a phase, just push the release.

An acquisition phase is made of 2 parts: criminal record snoring sounds, then record quiet. The second set off is equally influential as the first extraordinary, as IT will enable the ESP32 to learn to make out eupneic sounds from the background sound. If you intend to use the sensor at night, pay attention to show very scummy background at this moment.

The 2 parts last 3 seconds each. There is a 3 seconds holdup in between. Each recording part will save 10 spectra in 16 frequency bands, plus one safe volume data, for a total of 20 * (16+1) = 340 data.

The OLED screen door displays all instructions: push release to record (with the number of recording phases thusly far), make snore sound, remain silent, etc.

When you decide to stop recording, just unplug the ESP32.

Change the parameters

During the recording phases, the OLED screen wish display the magnetic declination of the acoustic spectrum. As I aforesaid above, the sound spectrum is divided into 16 sub-bands. This can be changed in the params.h file:

// Frequency bands #define BANDS 16

However, this total moldiness be a business leader of 2, as required by the FFT functions. So you tooshie increase it to 32, for a Thomas More accurate sampling, at the monetary value of a larger dataset and yearner learning phase.

I recommend to run at least 20 Beaver State more acquisition phases, to provide a large dataset to the encyclopaedism program.

Step 5: Run the Network

Once the dataset is in the SPIFFS, the ESP32 needs to train on that, in order to learn to recognize snoring sounds from other sounds. This is the objective of this forward program.

Now create a irregular folder in your Arduino folder, named Learning_ESP32 and put inside the succeeding files: Learning_ESP32.ino, params.h, Tinn.h, init.h, train_test.h, and sound_functions.h. Likewise create a Data subfolder, which will equal used if you want to upload files on your ESP32.

Make a point that the FFT parameters are the same as those used in the recording phase. Both params.h files essential take up the Sami following lines:

// FFT parameters #define SAMPLES 256 #define MAX_FREQ 20 // kHz

Train, baby, train...

The training form is when the neural network uses the dataset you provided to optimise its parameters to best healthy the complex function that can interpolate within all the samples in the dataset. This is an iterative process. Information technology is initialized at random, thusly you may deliver distinct results for each one time you test it. If you are not satisfied away the current results, equitable run it again and you may get better results...

Upload the Learning_ESP32.ino cartoon on your ESP32 and let it run. The OLED screen displays all the instructions.

It prototypical looks for a saved network lodge. Course, there is none such file if it is the first gear prison term you run this sketch. Otherwise, flick the button (within 3 seconds) to load the saved mesh, and the encyclopaedism form will be skipped.

If you didn't push the button, the encipher launches the eruditeness phase. This is an robotlike process, made of the following stairs:

  • Read the dataset: the CRT screen displays the summary of the content of the dataset filing cabinet,
  • Create a network and civilize it on the dataset: the screen displays a progress bar,
  • When the training results are satisfying, depending on the parameters: the screen displays the learning results (number of errors, error rates).
  • The network is saved in the SPIFFS, for later consumption.
  • Then the course of study goes into the inference phase.

The 3 first off parts are done in the setup, the loop takes care of the last matchless.

Changing the learning parameters

The neural network's parameters are in the params.h file:

#define RATIO 0.8f         // ratio of training data vs. testing #define EPOCHS 6000        // number of training epochs #define NHID 40            // number of hidden neurons #define ACTIVATION RELU    // chosen energizing function of hidden layer #define Lot 50           // number of data used for training in all date of reference #define Lr 1.0f            // first learning rate #define ANNEAL 0.9999f     // rate of interchange of learning rate #define MAXERR 0.0005f     // stop training if error is inferior than this #define DETECT 0.9f        // detection threshold

You can playact with them to get better results. Keep the RATIO greater or adequate 0.7

The DETECT parametric quantity is used for the "inference phase".

Inference?

Inference is the phase where the capabilities learned during preparation are put to make for. This is presented in the figure below.

In short, during the inference phase, the ESP32 listens and tries to notice a snore from the circumferent sounds. It records and analyses the sound, displays the spectrum on the Organic light-emitting diode test, and whenever the sound looks like the image of a snore IT built during the learning phase angle, it displays an alert message.

The file Data.txt contains the dataset I registered. I add information technology so you can see what this information look like.

Step 6: What Else?

The rest depends on what you deficiency to do.

Note that the Snore-O-Meter is an "autonomous" device: you don't need your PC to pull through work. All the instructions and information are displayed on the OLED screen door. You can just plug a USB phone charger on the ESP32 and let it run...

Monitor with Thingspeak

One great possibility is to monitor your nap and detect snoring phases. With the help of ThingSpeak, it's quite simple to store snoring data (such as detective work multiplication) and imag your eupnoeic action on a bar graph. Thingspeak mechanically adds a timestamp to the data you upload on your account.

Soh just import the Thingspeak library for Arduino from Here (there are a few examples for ESP32), create an account and a dedicated channelize. Each sentence the inference detects a snoring sound, just send a '1' to your conduct, and you're done! Some basic instructions are available here. Below is an example code:

// Write to ThingSpeak. There are equal to 8 fields in a channel, allowing you to store raised to  // 8 diverse pieces of information in a channel.  // Here, we write to field 1 the economic value 1 : writeField(Channel, Field, Value, APIKey). int x = ThingSpeak.writeField(myChannelNumber, 1, 1, myWriteAPIKey);  if(x == 200) Music.println("Line update winning."); else Serial.println("Problem updating channel. Hypertext transfer protocol error code " + String along(x));

Atomic number 3 you cannot send data faster than at one time every 15 seconds, remember to wait for that hold up before doing another inference.

And then plot your data using the bar chart template.

Introducing 'Termina-Saw logs'

Back to the original idea: how to stop stertor? Well... your mental imagery is the limit. You can add a simple piezo doorbell on your breadboard to play annoying sounds when a snore is detected. Hopefully, this will disturb you sufficient to make you move unofficially and stop snoring. Simply it may as well disturb whoever share your bedroom...

To do this, you need a piezo doorbell such As the one above and add a few lines of encipher in the Learning_ESP32 file...

First insert these lines before the setup, in the definition block:

const int buzzerPin = 18;  // put Hera the GPIO number for the buzzer<br>int channel = 0; int resolution = 8;

The ESP32 handles the PWM a bit other than than the different Arduino boards: here we define the GPIO number connected to the IO of the buzzer, and choose a channel number and its declaration (in bits) for the PWM.

Then, come in these lines at the start of the setup:

              ledcAttachPin(buzzerPin, channel);<br>     ledcSetup(channel, 255, resolution);   int volume = 10;  // 0 < volume < 255   ledcWrite(channel, volume);

This links the pin number to the channel, and defines the transport's properties.

Finally, create a function that will act as a sound:

void playSound () {<br>  for (int freq = 300; freq < 6000; freq = freq + 50) {     ledcWriteTone(channel, freq);     delay(5);   } }

This function potty be put before the setup. You can modify it if you like. then, all you suffer to do is to add the following line in the eyelet:

playSound ();

just in between these 2 lines:

expose.display();<br>delay(400);

Some other possibility is to use a DFplayer MP3 module that will play some proper music that you leave unconsciously learn as an alarm to move on your English... Less disturbing, merely maybe inferior efficient.

Why not make a robotic arm, triggered by the ESP32, to slap your face when the snore is detected?

You'll call that TERMINA-SNORE... OK, maybe that's a little likewise complex.

Disclaimer: This was done lone for fun. I single did this to demonstrate that IT is possible to run artificial word applications on an ESP32. It quite nicely worked for me, but I can't control that IT will work with anybody else.

I am currently impermanent on a more drug user friendly way of running play Artificial insemination along the ESP32, namely a library. This will enable everyone to create their own Army Intelligence applications for their own purpose.

Receive playfulness anyway... and share your results.

Cost the First to Share

Recommendations