Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Horizontal Connecting Lines in Charts for Persistence Strategy everyChange #3662

Open
dilyanpalauzov opened this issue Jun 18, 2023 · 10 comments · May be fixed by #4610
Open

Use Horizontal Connecting Lines in Charts for Persistence Strategy everyChange #3662

dilyanpalauzov opened this issue Jun 18, 2023 · 10 comments · May be fixed by #4610
Labels
enhancement An enhancement or new feature of the Core

Comments

@dilyanpalauzov
Copy link
Contributor

An item maps to the power consumption of a pump. I persist changes on the power with strategy=everyChange. In a sitemap I insert

Chart item=e3_power period=4h refresh=60000

Then this URL is called http://openhab/chart?items=e3_power&period=4h&theme=bright&t=1687084184925 which produces

chart4h

The problem is that from the persisted data dots are created and these dots are then connected using tilted lines. E.g. if from 8:00 to 8:59:20 the consumption was zero, and from 8:59:20 to 9:00:00 the consumption was 1000, there is no stored information that at e.g. 8:50:00 the consumption was zero, because the persistence strategy is everyChange. The graph shall nevertheless make evident, that at 8:50:00 the consumption was zero.

Please make it possible to create charts, where the lines connecting the individual dots are only horizontal. I understand that tilted lines have their merits to visualize interpolated data, but in the concrete use case with everyChange persistence strategy the diagram is inaccurate. When the tilted lines are very close to each other (the pump is turned on and off in short periods of time) it is hardly possible to read from the graph, whether the device was all the time on, or it was also off in the meantime. This gets even more complicated if I take a longer period, like 12h.

My suggestion is: when the persistence strategy for an item is everyChange use in the charts only horizontal lines (no tilted lines). With other persistence strategies keep the current logic.

The above is written having sitemaps in mind. I have not tried yet all possible options in “Main UI”. In any case, in “Main UI” shall also exist an option to use horizontal lines to connect the dots, if there is no such option yet.

I use OpenHAB 3.4.2

@dilyanpalauzov dilyanpalauzov added the enhancement An enhancement or new feature of the Core label Jun 18, 2023
@dilyanpalauzov
Copy link
Contributor Author

Just showing dots for the present data, avoiding any connecting lines would also work, as I can see what is stored.

@rkoshak
Copy link

rkoshak commented Jun 19, 2023

First of all, charting has not knowledge about the persistence strategy. All it sees are points.

The charting servlet that generates the charts for the sitemap Chart element is very basic and simple. I don't know how feasible it is to actually provide any sort of configuration like that.

The standard approach to solve this issue is to add a periodic strategy in addition to the everyChange strategy. That will add points to the graph in between the changes and flatten out the chart.

For even more configurability there is Grafana (there are lots of tutorials on the forum).

My suggestion is: when the persistence strategy for an item is everyChange use in the charts only horizontal lines (no tilted lines). With other persistence strategies keep the current logic.

Since charting knows nothing about the persistence strategy, this idea is not possible.

@dilyanpalauzov
Copy link
Contributor Author

Thanks for your answer.

The standard approach to solve this issue is to add a periodic strategy in addition to the everyChange strategy. That will add points to the graph in between the changes and flatten out the chart.

I want to avoid this for performance reasons. In this concrete case it is even not predictable what the period shall be, perhaps 30seconds, and this will create a lot of unnecessary load.

I actually considered inserting (persisting) the old value from within openhab-js, at the moment when the new value arrives, but as far as I could see it is not possible to persist a value from the for the past (1second ago, or alike)

@rkoshak
Copy link

rkoshak commented Jun 19, 2023

What performance problems are you having where configuring these few Items to save every minute or every five minutes makes any difference?

The amount of time for how often to save the value depends on the time ranges of the charts desired.

I actually considered inserting (persisting) the old value from within openhab-js, at the moment when the new value arrives, but as far as I could see it is not possible to persist a value from the for the past (1second ago, or alike)

This is an approach that could work but you'd need to use a proxy Item. Lets say we have an Item named "Raw" linked to the Channel and another names "Proxy". Note this won't work with rrd4j, but rrd4j requires an everyMinute strategy anyway so you wouldn't see this problem in the first place.

  • Exclude Raw from persistence or just ignore it, you'll use Proxy on the sitemap and for charting
  • Configure Proxy with an everyChange strategy
  • Trigger a rule when the "raw" Item linked to the Channel changes. In this rule, first call Proxy.persist() (or the equivalent in your language of choice) and then call Proxy.postUpdate(Raw.state) (or the equivalent in your language of choice)

@dilyanpalauzov
Copy link
Contributor Author

The system has already a load average of 1.2 (120%).

The simplest solution would be if the dots are connected only with horizontal lines, as an option.

I will give that proxy in the next days a try.

@rkoshak
Copy link

rkoshak commented Jun 19, 2023

I doubt it. A system load tells one nothing about what is causing the CPU to be stuck waiting (that's what the load means). Without out other evidence, saving a couple hundred extra data points every few minutes is unlikely to impact that one bit. It's at best a stab in the dark or just an assumption.

And this assumes that this would be simple to implement which I am assuming it is not or else this would have been implemented by now. Keep in mind that the charting servlet has been around for at least ten years. It's not new code and has heavy dependencies on third party libraries.

@dilyanpalauzov
Copy link
Contributor Author

Using a Proxy and Raw item, as suggested above, is a functioning workaround.

Irrespective of this I fetch data as fast as possible from a device, with a delay in between, so that my system has a load average around 1.0. That is, if I can reduce any forms of load, so that the system has more free CPU cycles, then I will decrease the aforementioned delay.

@dilyanpalauzov
Copy link
Contributor Author

This repesitory contains the word diagonal on two places: in org.openhab.core.io.rest.core/src/main/java/org/openhab/core/io/rest/core/internal/persistence/PersistenceResource.java and in org.openhab.core.ui/src/main/java/org/openhab/core/ui/internal/chart/defaultchartprovider/DefaultChartProvider.java. At least the latter inserts a point just before the current moment, in order to avoid tilted lines, thus to have only horizontal conneting lines.

@schup011
Copy link

schup011 commented Feb 12, 2025

My suggestion would be to offer a chart option like

linestyle=connecting (default)
linestyle=horizontal

in order to enable this behavior. As @dilyanpalauzov pointed out, this is already implemented for OnOff types:

           // For 'binary' states, we need to replicate the data
           // to avoid diagonal lines
           if (state instanceof OnOffType || state instanceof OpenClosedType) {
               xData.add(Date.from(historicItem.getTimestamp().toInstant().minus(1, ChronoUnit.MILLIS)));
               yData.add(convertData(state));

So the task would only be to insert the option and to distribute it to the addItem method where these code lines are from.

@schup011
Copy link

BTW, I find it absolutely no good idea to add additional data to the persistence data in order to display it better. This information is redundant and only solves the issue partly as there will be always a diagonal line between the last update and the next new state.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement An enhancement or new feature of the Core
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants