My electric water heater consumes a lot of electricity, and it does so at times when it is most expensive. This is because I have dynamic prices which change every hour and I shower at an expensive time. And even if a had a high&low tariff, the problem would still exist.
There is more than one way to solve this problem, and I prefer one that is technological in nature.
Table of Contents
- Solution? A smart switch!
- The big picture
- Small computer
- Find rates website
- Extracting data
- Save webpage to file
- Find the table
- Make a switch decision
- Output signal
- How to drive the relay
- Putting it all together
- Finishing the software
- Testing the switch in the shed first
- Auto-start the script upon reboot
- Places to go from here
Solution? A smart switch!
Because I already have flexible rates for my electric car, I need to stick to that. This forces the solution to be a smart switch. This switch will only power the electric water heater when electricity is relatively cheap.
The big picture
The computer has to download the dynamic prices from the internet once a day. This is easiest to do in Python. That makes the cheapest capable device a Raspberry Pi. But any old laptop should also suffice.
In the future, when building more switches, we can have a central device or website maintain the hourly prices for electricity. Then an ESP32 like device could control the switch and make a big improvement on all fronts.
Find rates website
Although the graph of my provider shows me when prices are high or low, it’s a graph and not very useful to a computer. A table would be better, that’s why we need to find a site that has those.
I choose to use the website https://www.entsoe.eu/, it has hourly price data for almost the entire EU (which is useful for me) and provides the data in both forms.
After finding the right table of data, I want to extract the data so I can use it. For this we need the page’s source code.
This source code might not look very useful, but it is structured and it is textual. That means that we can search it for our data.
My first attempt at this was to fetch the page using a web-request in Python, only to not find the data I was looking for. As it turns out, most websites that present data, load it in after the fact using AJAX and similar techniques.
But then there is Selenium, a Python package that lets us automate an actual browser.
Follow the instruction below to install Selenium on the Raspberry Pi. Or find yourself a good tutorial on it depending on your device.
sudo apt-get install chromium-driver pip3 install selenium
Save webpage to file
from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.common.keys import Keys # The web adress where we can find the table with prices entsoe_url = "https://transparency.entsoe.eu/transmission-domain/r2/dayAheadPrices/show?name=&defaultValue=false&viewType=TABLE&areaType=BZN&atch=false&dateTime.dateTime=20.10.2022+00:00|CET|DAY&biddingZone.values=CTY|10YNL----------L!BZN|10YNL----------L&resolution.values=PT60M&dateTime.timezone=CET_CEST&dateTime.timezone_input=CET+(UTC+1)+/+CEST+(UTC+2)" def retrieve_prices(): # Create the automated browser object browser = webdriver.Chrome() # Have it go to the url annd return the entire content of the page browser.get(entsoe_url) html = browser.page_source # Write html content to disk for testing and debugging if needed. with open('content.txt', 'w') as f: f.write(html) # And close the browser browser.quit() if __name__ == '__main__': price_list = retrieve_prices()
This short script will create a browser object, open the web-page en retrieve the entire content. This content will then be written to a file called ‘content.txt’.
Now we have the content stored in a file. No need to keep hitting the web-server while we are debugging.
Find the table
The first price in the table is 74.50, so I open ‘content.txt’ and go look for it.
Because prices are different every day, I can’t use that to find the beginning of the table. We have to look at text in front of the price that looks kinda specific for that part but also returning every day.
This word only start appearing at the start of the table, so if we can first the first one, our table starts there.
To keep things simple, I create a new script to work with the ‘content.txt’ file from here on. Later on I will combine them into the final script.
# Finds a substring in a string and cuts of everything before that occurrence. def goto(haystack, needle): position = haystack.find(needle) if position == 0: return haystack return haystack[position:] def retrieve_prices(): # Read the content of the file. with open('content.txt', 'r') as f: html = f.readlines() html = ''.join(html) html = html.replace('\n', '') # Find the start of the hours column html = goto(html, 'DayAheadPricesMongoEntity') print(html[0:250]) if __name__ == '__main__': price_list = retrieve_prices()
A Regex, short for regular-expression, is a fancy way to describe and look for text.
This is what the text around a prices looks like in regex format. The
(.*?) part is a wildcard match where we expect a price to be.
I expanded the script to look for this expression.
import re # Finds a substring in a string and cuts of everything before that occurrence. def goto(haystack, needle): position = haystack.find(needle) if position == 0: return haystack return haystack[position:] def retrieve_prices(): # Read the content of the file. with open('content.txt', 'r') as f: html = f.readlines() html = ''.join(html) html = html.replace('\n', '') # Find the start of the hours column html = goto(html, 'DayAheadPricesMongoEntity') # Define what we are looking for pattern = 'data-view-detail-link<\/span>"><\/span>(.*?)<span' matches = re.findall(pattern, html) # Display all the matches for m in matches: print(m) # Make sure there are exactly 24 prices assert(len(matches) == 24) if __name__ == '__main__': price_list = retrieve_prices()
Make a switch decision
Now we have prices, we can actually make a decision to turn the switch on or off. I added to the second script, the function now returns a list of prices and the main function makes the decision.
# Return as numbers return [float(i) for i in matches] if __name__ == '__main__': old_day = -1 old_hour = -1 prices = None # Run forever while True: now = datetime.datetime.now() # New day? Download new prices if now.day != old_day: prices = retrieve_prices() old_day = now.day # New hour? Check if price is below average if now.hour != old_hour: average_price = sum(prices) / len(prices) if prices[now.hour] < average_price: print('Switch on') else: print('Switch off') old_hour = now.hour # Sleep for a while time.sleep(10)
The logic of the script is very simple. New day? Then download new prices. New hour? Then check if current price is above or below average and switch accordingly. These conditions will be checked every 10 seconds, forever.
Now I have create a third script, that does nothing but flip a digital output pin on the Raspberry Pi on and off every 5 seconds. This will be very useful during the building of the hardware part of the switch.
import time import RPi.GPIO as GPIO switch_pin = 40 # GPIO21 GPIO.setmode(GPIO.BOARD) GPIO.setup(switch_pin, GPIO.OUT) switch = False while True: time.sleep(5) switch = not switch if switch: GPIO.output(switch_pin, GPIO.HIGH) print('On') else: GPIO.output(switch_pin, GPIO.LOW) print('Off')
Small note about the script.
switch = not switch does not mean I doubt its existence. It basically means, if its false, then make it true and the other way around. This makes it go On-Off-On-Off-etc.
That is a lot of wire just to measure the voltage, but it does the job.
How to drive the relay
The IO pins on the Raspberry are not powerful enough to drive the relay directly, so I need a man in the middle.
I created this circuit on a breadboard and hooked it up between the Raspberry and the relay switch.
This circuit actually works, and with this heavy relay I can switch loads up to 40 amps. That is more than enough to switch my water heater.
Now that I saw this working, I feel confident enough to transfer the circuit over to some proto-board.
For the first version of a product, the size is very acceptable to me.
Putting it all together
I went to the store and got myself two extension-cords. One I kept intact to power the Raspberry and the second one is the switched one. I worked open the cable and cut one of the power wires. Then I soldered the ends before I screwed them into the relay.
With the Raspberry still loaded with the testing script, the light went on and off with an interval of a few seconds.
Finishing the software
The three separate scripts can be lumped together fairly easy, all the difficult parts are there to re-use. One thing has to be added to the script however. The URL we used for testing was a static one, with a fixed date in it. I added some code to make the URL always for today’s date.
Testing the switch in the shed first
I could now use the switch for my water heater, but that is two floors up in my house and out of sight. I think a better option would be to test the switch for a few days in my shed, switching a lamp, just to see how it manages over time. Just in case it creates a fire or worse, causing the water-heater to give me cold water while in the shower.
Auto-start the script upon reboot
- Working on it
- Working on it
Places to go from here
- Working on it