I currently have a program that takes the data from an accelerometer, specifically Adafruit's ADXL345, and writes it to a JSON at a sampling rate defined by the user, and then, after a period of time also defined by the user, sends this JSON file to ThingSpeak. I can change the sampling rate by changing the variable update_interval.

This program works when I set update_interval = 1. It is able to successfully write the accelerometer data to the JSON file once per second and then upload this file to ThingSpeak, in this instance, every 5 seconds.

However, accelerometer data of 1 Hz isn't of much use to me so I want to increase the sampling rate. When I set update_interval = 0.01 it is able to successfully write the accelerometer data to the JSON file as far as I can tell, but it is then unable to upload the JSON file to ThingSpeak. When it attempts to upload the JSON file it gives the error code 400, which means the upload request can't be fulfilled due to "bad syntax" [1]. I don't understand why this would be occurring as I am not altering the code pertaining to the upload of the JSON file. If someone could help me out that would be much appreciated. I am but a simple mechanical engineer trying to learn python for the first time for a project.

[1] .html

Full Code:

#1. Import necessary libraries for the script
import sys
import json
import time
import os
import psutil
import requests
import busio
import board
import adafruit_tca9548a
import adafruit_bme680
import adafruit_adxl34x
import adafruit_sgp40
#import adafruit_ads1x15.ads1015 as ADS
import paho.mqtt.publish as publish
import string
import numpy as np

#from adafruit_ads1x15.analog_in import AnalogIn

i2c = busio.I2C(board.SCL, board.SDA)
tca = adafruit_tca9548a.TCA9548A(i2c)

#Define what channel each of the sensors are plugged into
bme680 = adafruit_bme680.Adafruit_BME680_I2C(tca[0])
accelerometer = adafruit_adxl34x.ADXL345(tca[1])
sgp = adafruit_sgp40.SGP40(tca[2])
#ads = ADS.ADS1015(tca[3])

#Set the sampling rate of the accelerometer to 800 Hz
accelerometer.data_rate = adafruit_adxl34x.DataRate.RATE_100_HZ

bme680.oversample_humidity = 2
bme680.oversample_temperature = 2
bme680.oversample_pressure = 2
bme680.oversample_filter = 2

ADC_CHANNEL = 0
GAIN = 1

#2. Define global variabesl that track the last connection time and last update time. Define time intervals to update the data, and post the data to ThingSpeak
last_connection_time = time.time()  #Track the last connection time
last_update_time = time.time()      #Track the last update time
posting_interval = 5               #Post data once every 5 seconds
update_interval = 0.01              #Write data to a JSON file in a defined amount of time

#3. Deinfe you ThingSpeak write API key and channel ID settings, along with the ThingSpeak server settings
write_api_key = "Y1S2TKIM4XTQHESF" 
channel_ID = "2729406"
url = "/" + channel_ID + "/bulk_update.json" #ThingSpeak server settings
message_buffer = []

#4. Define the function httpRequest that sends data to ThingSpeak and prints the response code from the server. 
#The response code 202 indicates that the server has accepted the request and will process it.
def httpRequest():
    #Function to send the POST request to ThingSpeak channel for bulk update.
        global message_buffer
        bulk_data = json.dumps({'write_api_key':write_api_key,'updates':message_buffer}) #Format the json data buffer
        request_headers = {"User-Agent":"mw.doc.bulk-update (Raspberry Pi)","Content-Type":"application/json","Content-Length":str(len(bulk_data))}
        """
        # Save JSON data to a file before sending
        timestamp = int(time.time())
        filename = f"data_to_thingspeak_{timestamp}.json"
        with open(filename, "w") as file:
            file.write(bulk_data)
        print(f"Data saved to {filename}")
        """
    #Make the request to ThingSpeak
        try:
            print(request_headers)
            response = requests.post(url,headers=request_headers,data=bulk_data)
            print (response) #A 202 indicates that the server has accpeted the request
            #input("Press enter to continue...")
        except e:
            print(e.code) #Print the error code
        message_buffer = [] #Reinitialize the message buffer
        input("Press enter to continue...")
        global last_connection_time
        last_connection_time = time.time() #Update the connection time
        
#5. Define the fucntion getData that returns the data
def getData():
    #Function that returns the data from the array of sensors
            
            
            #bme680
            humidity = bme680.relative_humidity
            temperature = bme680.temperature
            pressure = bme680.pressure
            gas_resistance = bme680.gas
            #sgp40
            #voc = sgp.raw
            #adxl345
            acc_x, acc_y, acc_z = accelerometer.acceleration
            #acc_x, acc_y, acc_z = acceleration_rms()
            #spw2430 and ADS1015
            #channel = AnalogIn(ads, ADS.P0)
            #mic_value = channel.value
            return humidity,temperature, pressure, gas_resistance, acc_x, acc_y, acc_z
#6. Define the function updatesJson to continuosuly update the message buffer every 15 seconds

def updatesJson():
    #Function to update the message buffer every 1 second with data.
    #And then call the httpRequest function every 1 minute.
        global last_update_time, last_connection_time
        
        if time.time() - last_update_time >= update_interval:
            # Create a message with sensor data
            message = {}
            message['delta_t'] = int(round(time.time() - last_update_time))
            humidity, temperature, pressure, gas_resistance, acc_x, acc_y, acc_z = getData()
            #message['field1'] = humidity
            #message['field2'] = temperature
            #message['field3'] = pressure
            #message['field4'] = gas_resistance
            message['field5'] = acc_x
            message['field6'] = acc_y
            message['field7'] = acc_z
            #message['field8'] = mic_value
            global message_buffer
            message_buffer.append(message) #Add to buffer
            print(f"Data added to buffer: {message}") #Debugging output
            last_update_time = time.time() #Update last update time here
    # If posting interval time has crossed 2 minutes update the ThingSpeak channel with your data
        if time.time() - last_connection_time >= posting_interval:
                httpRequest()
                last_update_time = time.time()
                
#7. Run an infinite loop to continously call the function updatesJson every 15 seconds.
    
if __name__ == "__main__": #To ensure that this is run directly and does not run when imported
    while True:
            #If update interval time has crossed 1 second upates the message buffer with data
        if time.time() - last_update_time >= update_interval:
            updatesJson()

I have tried playing around with various other sampling rates to see if there are others that work as well. When I make update_interval = 0.5 the code is able to successfully upload the JSON file to ThingSpeak. However, if I increase the sampling rate, making update_interval smaller, it also no longer works. I have looked elsewhere for solutions but to little avail.

I currently have a program that takes the data from an accelerometer, specifically Adafruit's ADXL345, and writes it to a JSON at a sampling rate defined by the user, and then, after a period of time also defined by the user, sends this JSON file to ThingSpeak. I can change the sampling rate by changing the variable update_interval.

This program works when I set update_interval = 1. It is able to successfully write the accelerometer data to the JSON file once per second and then upload this file to ThingSpeak, in this instance, every 5 seconds.

However, accelerometer data of 1 Hz isn't of much use to me so I want to increase the sampling rate. When I set update_interval = 0.01 it is able to successfully write the accelerometer data to the JSON file as far as I can tell, but it is then unable to upload the JSON file to ThingSpeak. When it attempts to upload the JSON file it gives the error code 400, which means the upload request can't be fulfilled due to "bad syntax" [1]. I don't understand why this would be occurring as I am not altering the code pertaining to the upload of the JSON file. If someone could help me out that would be much appreciated. I am but a simple mechanical engineer trying to learn python for the first time for a project.

[1] https://www.mathworks/help/thingspeak/error-codes.html

Full Code:

#1. Import necessary libraries for the script
import sys
import json
import time
import os
import psutil
import requests
import busio
import board
import adafruit_tca9548a
import adafruit_bme680
import adafruit_adxl34x
import adafruit_sgp40
#import adafruit_ads1x15.ads1015 as ADS
import paho.mqtt.publish as publish
import string
import numpy as np

#from adafruit_ads1x15.analog_in import AnalogIn

i2c = busio.I2C(board.SCL, board.SDA)
tca = adafruit_tca9548a.TCA9548A(i2c)

#Define what channel each of the sensors are plugged into
bme680 = adafruit_bme680.Adafruit_BME680_I2C(tca[0])
accelerometer = adafruit_adxl34x.ADXL345(tca[1])
sgp = adafruit_sgp40.SGP40(tca[2])
#ads = ADS.ADS1015(tca[3])

#Set the sampling rate of the accelerometer to 800 Hz
accelerometer.data_rate = adafruit_adxl34x.DataRate.RATE_100_HZ

bme680.oversample_humidity = 2
bme680.oversample_temperature = 2
bme680.oversample_pressure = 2
bme680.oversample_filter = 2

ADC_CHANNEL = 0
GAIN = 1

#2. Define global variabesl that track the last connection time and last update time. Define time intervals to update the data, and post the data to ThingSpeak
last_connection_time = time.time()  #Track the last connection time
last_update_time = time.time()      #Track the last update time
posting_interval = 5               #Post data once every 5 seconds
update_interval = 0.01              #Write data to a JSON file in a defined amount of time

#3. Deinfe you ThingSpeak write API key and channel ID settings, along with the ThingSpeak server settings
write_api_key = "Y1S2TKIM4XTQHESF" 
channel_ID = "2729406"
url = "https://api.thingspeak/channels/" + channel_ID + "/bulk_update.json" #ThingSpeak server settings
message_buffer = []

#4. Define the function httpRequest that sends data to ThingSpeak and prints the response code from the server. 
#The response code 202 indicates that the server has accepted the request and will process it.
def httpRequest():
    #Function to send the POST request to ThingSpeak channel for bulk update.
        global message_buffer
        bulk_data = json.dumps({'write_api_key':write_api_key,'updates':message_buffer}) #Format the json data buffer
        request_headers = {"User-Agent":"mw.doc.bulk-update (Raspberry Pi)","Content-Type":"application/json","Content-Length":str(len(bulk_data))}
        """
        # Save JSON data to a file before sending
        timestamp = int(time.time())
        filename = f"data_to_thingspeak_{timestamp}.json"
        with open(filename, "w") as file:
            file.write(bulk_data)
        print(f"Data saved to {filename}")
        """
    #Make the request to ThingSpeak
        try:
            print(request_headers)
            response = requests.post(url,headers=request_headers,data=bulk_data)
            print (response) #A 202 indicates that the server has accpeted the request
            #input("Press enter to continue...")
        except e:
            print(e.code) #Print the error code
        message_buffer = [] #Reinitialize the message buffer
        input("Press enter to continue...")
        global last_connection_time
        last_connection_time = time.time() #Update the connection time
        
#5. Define the fucntion getData that returns the data
def getData():
    #Function that returns the data from the array of sensors
            
            
            #bme680
            humidity = bme680.relative_humidity
            temperature = bme680.temperature
            pressure = bme680.pressure
            gas_resistance = bme680.gas
            #sgp40
            #voc = sgp.raw
            #adxl345
            acc_x, acc_y, acc_z = accelerometer.acceleration
            #acc_x, acc_y, acc_z = acceleration_rms()
            #spw2430 and ADS1015
            #channel = AnalogIn(ads, ADS.P0)
            #mic_value = channel.value
            return humidity,temperature, pressure, gas_resistance, acc_x, acc_y, acc_z
#6. Define the function updatesJson to continuosuly update the message buffer every 15 seconds

def updatesJson():
    #Function to update the message buffer every 1 second with data.
    #And then call the httpRequest function every 1 minute.
        global last_update_time, last_connection_time
        
        if time.time() - last_update_time >= update_interval:
            # Create a message with sensor data
            message = {}
            message['delta_t'] = int(round(time.time() - last_update_time))
            humidity, temperature, pressure, gas_resistance, acc_x, acc_y, acc_z = getData()
            #message['field1'] = humidity
            #message['field2'] = temperature
            #message['field3'] = pressure
            #message['field4'] = gas_resistance
            message['field5'] = acc_x
            message['field6'] = acc_y
            message['field7'] = acc_z
            #message['field8'] = mic_value
            global message_buffer
            message_buffer.append(message) #Add to buffer
            print(f"Data added to buffer: {message}") #Debugging output
            last_update_time = time.time() #Update last update time here
    # If posting interval time has crossed 2 minutes update the ThingSpeak channel with your data
        if time.time() - last_connection_time >= posting_interval:
                httpRequest()
                last_update_time = time.time()
                
#7. Run an infinite loop to continously call the function updatesJson every 15 seconds.
    
if __name__ == "__main__": #To ensure that this is run directly and does not run when imported
    while True:
            #If update interval time has crossed 1 second upates the message buffer with data
        if time.time() - last_update_time >= update_interval:
            updatesJson()

I have tried playing around with various other sampling rates to see if there are others that work as well. When I make update_interval = 0.5 the code is able to successfully upload the JSON file to ThingSpeak. However, if I increase the sampling rate, making update_interval smaller, it also no longer works. I have looked elsewhere for solutions but to little avail.

• python
• json
• raspberry-pi

1 Answer 1

There are some speed limits to consider. There is a limit to how fast you can sample the accelerometer (3200Hz) based on the datasheet's specs. There is also measurable turn around time for the requests library to reach out over the internet from your device to the Thingspeak servers, authenticate, and then POST the http data, and send back a response. I just did a test using requests to POST to Thingsboard and measure the turn around time like this:

time1 = time.perf_counter()
response = requests.get(url, json=jsonPayload)
time2 = time.perf_counter()
print('Turn around time to Thingspace: ' + str(time2-time1))

Results:
Turn around time to Thingspace: 0.325515782000366
Turn around time to Thingspace: 0.3308629659986764
Turn around time to Thingspace: 0.3106492549995892
Turn around time to Thingspace: 0.29507084600118105
Turn around time to Thingspace: 0.28653189999931783
Turn around time to Thingspace: 0.32124968000061926

The results averaged about 300ms. So in my case, the maximum rate I can post JSON files to the server is about 3Hz. Also keep in mind as the accelerometer sampling rate increases, the JSON file gets larger, faster. Transmitting larger files will increase the transmit time, decreasing the rate at which you can POST.