Author Archives: Greening Me

Washing on sunshine

As regular readers will recall I’ve recently updated my battery controls with solar predication. For some years now my battery storage has automatically charged during sunshine and later discharged in the evening, and over the last year I added the HEMS to manage buying electricity including for the battery when the electricity cost was cheapest as there’s little solar in the winter, but now the HEMS has the ability to automatically adjust how much power is bought from the grid depending on how much output is expected from the solar panels in order to deliver a charged battery by the end of the day when the electricity price rises significantly. The battery charging can thus swing from all solar to all grid and all points in between entirely automatically based on the solar forecast.

Predicted solar output for March 16th to 18th as of the evening of March 15th.

Now however I’ve also added the capability to adjust when the wet goods – dishwasher and washing machine – run as a function of expected solar panel output.

As with the battery controls, I don’t attempt to match generation half hour by half hour with device operation because of uncertainty around how precisely a solar forecast from the evening before will match actual solar production up to twelve hours later. Instead my solar algorithm extracts the number of hours of significant solar production and the earliest start time of that production. For the battery that information is used to modify the number of hours of battery charging to be bought from the grid and the end of the window during which those hours of charging may be bought. Now I’ve updated the controls for the wet goods similarly.

The existing wet good controls look to start the machine at the time when the electricity cost for the cycle is cheapest within a time window set by the user, so the user defines the earliest and latest acceptable start times and the algorithm finds the cheapest start time within that window. The updated wet goods controls assess the number of hours of solar charging available and if both (i) the window exceeds a certain size and (ii) the user’s time window includes the solar window then the user window is narrowed to start at the start of the solar window. The resulting start time between the start of the solar window and the end of the user window may not be absolutely the cheapest grid cost but my assumption is that the solar contribution (which could be up to 100%) will in practice make this the cheapest grid cost as any power needed from the grid will be at a relatively good price. If this start time in the solar window does not reflect absolutely the cheapest grid cost then a second start time may also be identified which is the cheapest grid cost.

HEMS schedule with start times for wet goods within the solar window.

In the above example the start times are both within the solar window and the cheapest energy price, so no cheaper alternative is also offered. Start times within the solar window tend to be in the afternoon as the grid energy costs are lower. This also improves the probability that the battery may briefly discharge if the total load exceeds the solar output. However in the above example the energy price is so low that the battery is also force-charging (it’s dark green) so any surplus demand will come from the grid.

HEMS schedule with start times both inside and outside the solar window

In the above example the cheapest time to run each cycle from the grid is at night, although given the availability of solar during the day then any small saving in grid costs at night is very likely wiped out by the ability to run some (if not all) of the cycle during the day from the solar surplus rather than the grid. Both start times are available – the absolutely lowest grid cost and the lowest grid cost during the solar window.

It’s probably also worth mentioning the implications of running the appliance alongside the battery status being different colours:

  • Blue battery and state of charge being at or over 80% – appliance is prioritised over battery allowing battery to discharge to meet appliance needs as required – least likely to draw anything from grid (but likely highest cost to draw from grid)
  • Light green battery or (blue battery and state of charge being below 80%) – appliance is prioritised over battery charging alone (so battery may not discharge to support appliance) – a little more likely to draw anything from grid (but likely mid cost to draw from grid)
  • Dark green battery – battery charging and appliance are equal priority – most likely to draw something from grid (but likely cheapest cost to draw from grid, even to the point of being paid to draw from grid at times)

What happens without internet?

Today we are increasingly reliant upon the internet. During the current crisis when many are confined to home the internet provides opportunities unimaged to previous generations. Whether it’s online shipping, entertainment, using zoom or other online meeting tools to maintain family, society or church connections; the internet provides the means to supply us with our material needs, entertainment and social contacts in ways unimagined previously.

However our home, and other smart homes, may be particularly vulnerable should the internet become unavailable either in general or a particular service upon which we rely becomes unavailable. I thus thought that for this post I’d reflect upon the services our home relies upon for normal operation and what the impact of their absence would be. I will also reflected upon other failures of the service where such failures have been previously observed.

HEMS schedule for 11-12/04/2020
ServiceEffect if unreachableeffect of other observed failure
Solar forecastingIn unreachable when daily schedule being generated then schedule assumes no solar production like mid-winter.On April 7th I observed several hours around the middle of the day when there were no forecasted or estimated actuals reported, but the service continued to accept our measured readings. No impact on HEMS operation.
Agile electricity price data.If unreachable when daily schedule being generated then schedule carries over from prior. Schedule may be generated by manual initiation of the script when the required data becomes available.
If necessary operation of the car charger PLC could be suspended causing the car to charge at full power immediately and the car’s own timer used to set operating hours if needed.
If tomorrow’s price data is not yet published (i.e. it’s overdue), then the optimisation is performed using the most recent day’s price data. Since today will no longer be within the dataset provided by the solar forecast (as it’s no longer in the future) then no solar window will be found and so all the required number of hours will be bought assuming today’s (not tomorrow’s) prices, so the battery will still be charged but at a sub-optimal cost. The optimisation can be started manually once the overdue pricing data is available.
Immersun APIIf the real time generation data wasn’t available at all then solar car charging would be disabled. Car charging based on bought electricity price would continue.I have observed occasions when the real time data ceases to be updated (presumably because communications between immersun and cloud is lost) which then throws out the upload to the solar forecasting and the disabling of solar car charging at low generation levels. The forecasting site doesn’t seem to be phased by some erroneous data having never yet dropped below 0.96 correlation. The solar car charging has once been enabled later than it should which had some impact on battery state of charge, but was disabled slightly later via the immersun relay output.
An API is also used to switch the immersun into or out of holiday mode on days with significant periods of negatively-priced electricity. Unavailability of the API could leave the immersun locked in holiday mode and thus completely unable to heat water, or not in holiday mode when it should be causing self-consumption to continue when export would be more optimal to allow for negatively-priced import later. Vacation mode may be enabled or disabled manually via the immersun’s front panel to mitigate.
Powervault APILack of availability of the API would disable the ability to switch operating modes. This would leave the Powervault either charging from solar only or force charging, and may or may not permit discharge. The unit can be reset to normal state via repeated operation of front panel button which would disable scheduling but provide default solar operation.
WIFIPLUG APILack of availability would prevent the wet goods being turned on via API. Operation via the bushputton on the plug and indeed via Apple Homekit should remain.
Apple HomekitThe heating and other automations are run from Apple Homekit. My understanding is that the local service is provided from the two Apple TVs acting as hubs which should continue. Remote access via devices not in the home would be disabled.
Hildebrand APII have access to an API giving data from the HAN side of my smart meter and independent current measurement, but this is not currently used for control so no operational impact.
Impact assessment for loss of different cloud services

Overall I would conclude that there’s no significant issue here. The house would continue to be heated and appliances will still work. Any impact would be around energy consumption and cost only.

Some mitigation could be arrived at by:

  • resetting the Powervault storage battery to restore default normal operation,
  • manually starting or stopping vacation mode on the immersun’s front panel, and
  • suspending operation of the PLC on the car charger to enable car charging.

Tuning up for the performance

Last night rather unexpectedly my solcast solar irradiance data tuned itself. I use this data to predict the output of my solar panels and adjust what I buy from the grid in response. I had expected that tuning would happen eventually, but my understanding was that two month’s data was required, not the two week’s data that I had so far supplied.

Prior to tuning my predictions had looked like this..

solcast predictions before tuning

Although the system is clearly predicting output to a reasonable degree of accuracy, there are two obvious issues:

  1. The orientation of the array from due south seems a little off, as my array starts and ends generation earlier than the prediction suggesting the the orientation should be slightly more easterly in the model.
  2. The peak at a sustained 4 kW is overly optimistic. The panels can generate 4 kW according to my monitoring, but only relatively briefly and certainly not for more than an hour in March.

Nevertheless, the overall identification of better and worse days is clearly working.

solcast predictions after tuning

However after tuning both issues have been resolved. The measured and predicted curves match very closely. Note that the prior predictions have been updated by the tuning process, so March 26th which is included in both the images looks subtly different. Note also that my control is based on forecasts from the evening before, not the ‘1 hour ahead forecasts’ illustrated above. You may also observe that there is a discrepancy in the morning of April first, where the measured data is zero while the estimated actuals and forecasts are quite healthy, which arises as a result of a temporary failure of the immersun server from where the data is taken.

The ability to do tuning requires an upload of data. I upload generation data continuously at 5-minute intervals (the shortest allowed interval) which may explain the early availability of the tuned results. The script that I use to achieve this takes data indirectly from my ImmerSUN and is modified from this script. I achieved a 99% correlation which is pretty good. Subsequently it seems that the tuning takes place automatically each day.

Tonic solar, a little light music

As winter turned to spring my thoughts for my HEMS turned to thinking about how to adjust the operation of the HEMS in managing battery charging to account for anticipated solar production. Previously the HEMS was configured to buy a preset number of hours of charge from the grid each day, typically overnight when the power tends to be cheapest. Through 2019 as the seasons changed I periodically adjusted this figure to create headroom to store charge from the solar panels later in the day. However I would like to make this adjustment automatically day by day.

Late last year I came across solcast a website that predicts the output of solar panels. The user creates an account, describes their PV array (location, capacity and orientation), and can then download predictions via API.

Solar prediction for the next three days

The orange line shows the predicted output for the next few days, with the light grey area showing the confidence interval from 10 to 90%. As a prediction there’s a degree of uncertainty associated with the prediction as there is with a weather forecast. The 10% line shows that 1 day in 10 the output will be lower than the grey area, while the 90% line shows that 1 day in 10 then output will be higher than the grey area.

My original prediction were based on the orange line (the 50th percentile) where output was equally likely to be above or below this amount. However my risk on an incorrect prediction is not even. If I fail to buy enough power from the grid when the price is low then I risk paying up to 35p/kWh to buy when the price is high, whereas if I buy an unnecessary cheap kWh from the grid I may spend an unnecessary 5p/kWh on average. Thus I decided to take a more conservative position on risk to obtain the lowest cost position. I opted for a 20% risk, so 1 day in 5 I might underestimate my purchase from the grid reflecting the ratio of grid prices high:low. I estimated this 20th percentile assuming a normal distribution on the low side where the 20th percentile = 0.34 * 50th percentile + 0.66 * 10th percentile.

The process is as follows:

  1. download the data in a half-hourly format to be comparable to the half-hourly Agile electricity price data,
  2. calculate the 20th percentile from the 10th and 50th percentiles,
  3. count the number of half-hours in the 20th percentile data above a threshold that provides for charging the battery at full power,
  4. establish the earliest half hour when solar would charge the battery at full power,
  5. adjust the period for buying power from the grid by the period anticipated for solar charging (tomorrow a total of 7 hours is to be achieved by 6.5 hours from solar leaving 0.5 hours from the grid),
  6. adjust the end time for buying power from the grid to align with the earliest hour when the battery can charge at full power from solar.
HEMS schedule with battery behaviour modified for predicted solar generation

The result of these calculations can be seen above. One half-hour of lowest price battery charging has been identified overnight to meet the requirement for charging from the grid. Normal HEMS behaviour has also identified several other periods of grid charging at lower cost during the day, but these are not counted towards the target for buying from the grid due to the potential for a double count of solar and grid charging during the day. (In a similar manner there are multiple start times during the afternoon when the dishwasher or washing machine can be run on grid power more cheaply than the optimal overnight start time.)

No explicit command from the HEMS is required to enable proportional charging of the battery from the solar panels when there is a solar surplus as 2 of the 3 used battery operating modes (normal and charge-only) have this capability, and the third mode force charges the battery. If the battery is force charged during solar surplus then the source of the energy will of course be the solar panels, but any shortfall will be met from the grid.

modecharge behaviourdischarge behaviourcomment
NormalProportional to solar surplusProportional to shortfallUsed at high grid prices (today discharging enabled at > 8 p/kWh)
Charge onlyProportional to solar surplusNo dischargeUsed at mid grid price to save stored energy for period of higher grid price (between 5 and 8 p/kWh today)
Force chargeFull powerNo dischargeUsed at lower grid price to buy from grid (today buying at < 5p/kWh). If low price occurs at time with reasonable solar generation then use of solar output will happen automatically, with only any shortfall coming from the grid.
Battery behaviour in different operating modes

Smart Export Guarantee – FiT for purpose?

Solar PV installations like mine that are a few years old generally qualify for the UK’s Feed-in Tariff (FiT) which pays both for generation and notionally for export, while newer installations are covered by the Smart Export Guarantee (SEG). The older FiT scheme was universal in the sense that all larger electricity companies had to participate and they all paid the same rates, while with the newer scheme there’s still an obligation for larger companies to participate but the rates are all different. Older installations like mine can optionally swap the export component of the FiT for the SEG, but is that an attractive option?

SEG Payments by provider

SEG payments differ widely between providers so it’s worth shopping around.

My FiT export payment is currently 5.38 p/kWh on a deemed export basis, which means that, rather than measure actual export, it is assumed that half of my generation is exported. My electricity supplier Octopus offerers one of the best SEG rates at 5.5 p/kWh but that’s on the actual export, not the deemed export.

Monitoring data March 2019 – February 2020
AlternativeDescriptionEnergy exportedRate paidTotalComment
Baseline FiT2,098 kWh (50% of 4,196.1 kWh) 5.38 p/kWh£112.87
Scenario #1 Switch to SEG without other changes 647.1 kWh5.50 p/kWh£35.5968% reduction
Scenario #2Add disable water heating from solar to above.1,722.4 kWh (1,075.3 + 647.1 kWh)5.50 p/kWh£94.75
Provide equivalent water heating from gas1,075.3 kWh 3.2 p/kWh / 90% (£38.23)
Total£56.5250% reduction

Octopus Energy does also offer the alternative of a variable export rate based on wholesale prices, akin to their Octopus Agile import tariff, but for export. However it’s my belief that I would need a much larger battery than I have now (4 kWh) in order to benefit from this as it will always be generally better value to use that stored energy to avoid the early evening peak price period (up to 35 p/kWh) than to sell it back to the grid at a lower price and then need to buy more energy myself. If I had a bigger battery (both in terms of energy and power) then I could both meet my own needs and sell back to the grid.

Overall however I think that it’s clear that, with my current relatively small battery and deemed export tariff, I’m better off on the older FiT scheme than the newer SEG scheme even with one of the better-paying SEG providers.

Saving on electricity

I’ve been seeing a few online advertisements recently touting 70% savings on electricity through a combination of solar panels and battery storage. I’ve also been looking for a way to express my savings through my smart tariff so this seemed like a opportunity to try that.

My start point is a years data from my monitoring system..

Monitoring data for March 2019 to February 2020

I also went through a year of electricity bills (with slightly different start and end dates) concluding that my average purchased electricity cost was 7.08 p/kWh. Thus my average electricity costs (including solar) are on the right of the table below:

sourcequantityest unit priceEst totalmy uniT pricemY totalmy saving v. Est
Bought4,309 kWh15.75 p/kWh£678.677.08 p/kWh£305.08£373.59
Generated2,473 kWh15.75 pkWh£389.500.00 p/kWh£0.00£389.50
Total / Average6,783 kWh15.75 p/kWh£1,068.324.5 p/kWh£305.23£763.09
Comparison between my electricity cost and the UK average

If I look at the Energy Saving Trust’s assumptions as a baseline, they have the average UK cost of electricity as 15.75 p/kWh. If I’m paying an average 4.5 p/kWh for each kWh used with my combination of solar PV, storage battery and smart tariff then I’m paying 28.6% of the cost of someone who used the same amount of electricity bought at the average UK rate or saving 71.4% of electricity cost. To put it another way, I’m paying £305.23 for electricity that would have cost the average UK consumer £1,068.32 (on the left of the table above) – a saving of £763.09.

(The baseline assumption that someone would have used the same amount of electricity as me without my level of technology is a slight over-estimate as I flex water heating between gas and electricity since my bought electricity price is sometimes lower than my bought gas price causing me to substitute electricity for gas. Someone on a conventional electricity tariff and gas would never make that substitution as their gas would always be cheaper than their electricity, hence my electricity consumption is a little higher than someone who would be on a conventional electricity tariff.)

I’m also generating feed-in tariff due to the age of my system (approximately 4.5 years old) which would be £714.59 per annum at current rates, and making 1,075 kWh of hot water from surplus solar electricity which saves £38.22 in gas (the diverted / hot water saving in the screenshot above is based on a notional electricity price, not a gas price). Unless I’ve missed something that’s an annual return of £1,515.90 (£763.09 + £714.59 + £38.22).

In my previous post I estimated my investment at £8,670 so with an combined annual savings and revenue of £1,515.90 that’s a 17.5% return or a payback of 5.7 years. Previously I’d estimated 9 years including the battery, but this was without the benefit of the smart tariff. As we’ve now had the solar PV for 4.5 years that’s very promising, although as my return seems to be accelerating it will take more than 4.5 past years + 1.2 future years (total 5.7 years) to achieve payback.

The current 5.7 years to payback would have achieved payback in spring 2021 as the near bookend, while the prior 9 years would have been autumn 2024 as the far bookend. In practice I could not have achieved the lower bookend of 5.7 years, even had I invested in all the supporting technologies simultaneously, because I’m combining the legacy Feed-in Tariff (FiT) scheme for my solar PV with the Octopus Agile dynamic smart electricity tariff which started in February 2018,

The cost of smart

Discussion elsewhere prompted me to look into what I spent on what you might term my energy smart systems relating to electricity consumption, so I thought I’d document it here.

ItemDescription CostComment
1Solar photovoltaic system (4kW)£5,500Bundled with ImmerSUN.
2Powervault battery storage (4kWh)£2,000Free installation as part of UKPN trial.
3ImmerSUN management system with monitoring.£600Estimate based on today’s pricing.
4Remote-controlled car charger.£300Modified used charger from eBay. My own software.
5Raspberry Pi items to make HEMS£200My own software.
6Wet goods automation (WIFIPLUG x 2)£70
TOTAL£8,670

Prior analysis of items #1-#4 in pre-Agile days has suggested a total of 9 years to achieve payback on this investment through use of around 85% of the generated energy. Solar panels are potentially good for over 20 years operation, although I doubt the lead-acid batteries will still be operating for anything like that long.

The combination of item #5 with my Octopus Agile dynamic smart electricity tariff has resulted in my average bought electricity price being 7.75 p/kWh in 2019, about half the UK average. I suppose that I could make the same judgements and program items manually each day, but the HEMS significantly reduces my time commitment to achieve that.

Item #6 is my most recent addition. The sophistication of the algorithm combining the Agile tariff with a simple model of the cycle of each device is such that I would never achieve such a high quality result manually. However the saving is perhaps only a three pence each day so maybe £10 per year on my Agile tariff and thus 7 years to pay for the two smart plugs.

Much of this content is thus around 7 years to payback. The HEMS is potentially much quicker, but relies on having smart systems to control such as battery storage and car charger.

The ‘Appiest Days of My Life

One of the consequences of integrating a smart home is the large number of different apps, web portals and potentially sources of APIs involved. The ones I use include:

TitleAppPortalAPIPurposeComment
BrightYNYReads and stores consumption from smart meter.No price data for my tariff due to smart meter limitations.
EveY /3NNEve’s alternative to Home for all HomeKit accessories with additional functionality for Eve’s own devices.I prefer this to Home for editing rules.
I use Eve products mostly for central heating control.
HomeY /3NNApple’s own app for the HomeKit smart home ecosystem.Need to refer to device manufacturers own apps (such as Eve or WIFIPLUG) for some configuration and data.
HEMSNYN My own web portal to view HEMS schedule and status via Apache web-server on Raspberry Pi.
MyImmersunYYY /1Control of ImmerSUN power diverter.Available API provides some measurement and status data as per main screen of the app.
PowervaultNYY /2Control of Powervault storage system.Available APIs provide some user scheduling and status capability.
OctoWatchdogY /3YYFuture cost, and historic costs and consumption (30 prior days) from Octopus (electricity supplier).APIs provided by Octopus.
App developed by an enthusiast using Octopus APIs.
Octopus’s own web portal provides historic consumption but does not pair this with cost. Monthly statements show graph of consumption and cost for each day.
WIFIPLUGYNYControl and measurements from own brand smart plugs.Plugs also appear in Home and Eve apps.
I use for dishwasher and washing machine.

Notes to table:

  1. APIs not officially released. Reverse-engineered by an enthusiast and available on line.
  2. APIs not officially released. Used as part of a sponsored trial when I first got the battery and re-used by myself with some manufacturer support.
  3. iOS only. Not available for Android.

Some of these apps have similarities:

  • Both Bright and OctoWatchdog show whole of house energy consumption (and potentially cost) derived from the smart meter. However they have differences too. A smart meter sits on two networks: (i) the Wide Area Network (WAN) via which the meter communicates with the energy supplier and (ii) the Home Area Network (HAN) which links the devices in the home (electricity meter, gas meter, CADs/IHD and gateway). Bright connects to the HAN via small piece of hardware called a Glow Stick Wi-Fi CAD and collects its own data in real time and stores its own records of energy consumption in the cloud; while OctoWatchdog involves no extra in-home hardware, and takes data a day in arrears from Octopus not storing anything in the cloud itself. Bright’s USP is the real time consumption and current day’s data (neither of which OctoWatchdog supports), while OctoWatchdog’s USP is the availability of electricity price which isn’t available from the meter.
  • Both Eve and Home interact with all devices in the whole HomeKit ecosystem. Eve is best for creating rules and has more ability to configure Eve’s own devices, while Home is best for sharing access with family members. WIFIPLUG’s app is more limited only interacting with their own devices, and thus cannot see Eve or other HomeKit devices.
  • Both MyImmersun and WIFIPLUG apps, and the Powervault portal, allow configuration of their own manufacturer devices. They all have, for example, timer capability and data logging. MyImmersun is better for giving a whole-of-home view showing solar panel output and net input to house (so provides a more comprehensive energy monitor), Powervault shows no solar panel output but does give a view of whole-of-home, while WIFIPLUG provides only a view of the energy consumption of devices plugged in to the WIFIPLUGs.

HEMS2 – improving the breed

It’s now around a year since I first started with my HEMS. Initially it was managing just the charging of my electric car around the cheapest electricity prices, but subsequently I added water heating, storage battery control and, most recently, dish washer and washing machine.

That original HEMS was built around a used Raspberry Pi that I bought cheaply from a colleague but, while its processing power was perfectly adequate for the task (which isn’t at all demanding), it did have some mechanical issues. Firstly, the mounting holes for accessory cards (known as HATs – HArdware on Top) moved between generations of Raspberry Pi and the holes in my Pi for the standoffs did not align with those in the newer accessory card. Secondly, I never managed to find a case with the space for the HAT and the mains cables that it switches. The former resulted in the electrical connector between the Pi and the HAT taking much of the mechanical load, and an occasional loss of connectivity to the HAT.

These mechanical issues are resolved by the new HEMS which uses a third generation Raspberry Pi allowing use of proper standoffs and a new case with the depth to enclose the HAT too.

Relay HAT on Raspberry Pi 3 (without standoffs fitted or insulation layer).
HEMS2 (the black box) replacing my original HEMS

As previously two cables connect to the HEMS – a black USB cable bringing 5V power and a grey multi core cable that brings a live mains feed to the relays with three switched lives returning to the adjacent junction box. The relays control my less intelligent loads – the ImmerSUN and the car charger – while other more intelligent loads are controlled via APIs via Wi-Fi and the cloud. The whole assembly continues to be mounted on a board on the side wall of the airing cupboard. When modification is required the assembly can be lifted off two screw heads and laid on the floor.

All the software on both generations of HEMS is the same except for the scripts that interact with the relays – either to set or read status. I assume that the pinouts on the Pi must have changed between generations as an identical HAT card moved from the ic0 to the ic1 bus requiring a single digit change to each interacting command.

Raspberry Pi images are licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license.
Attribution: Efa at English Wikipedia

Refuse to Recycle?

Our local authority asks residents to separate a variety of materials for recycling (or landfill) as follows:

  • Glass in a dedicated plastic box.
  • Other recyclables (cardboard, metal, plastic etc) in disposable council-supplied pink sacks.
  • Kitchen waste (via a grey caddy) and garden waste in a green council-supplied wheelie bin.
  • Corrugated cardboard folded and packed in a corrugated cardboard box.
  • Textiles in a bag with a special council-supplied label.
  • Non-recyclables in a black bag.

It’s important to dispose of waste properly as some materials will take an extremely long time to break down naturally causing them to accumulate in the environment.

Decomposition times by material type.

Many of these things including steel and aluminium (or aluminum for our US readers) are not only greener but cheaper to recycle than make new ones.