problem display diacritical accents , custom sensor

Benoit_Mathieu · September 17, 2016, 4:12pm

Hello everybody,

I’ve just started to use OpenRemote and try to do a weather forecast. I take
forecasts weather via an http request. I store the xml received file in local directory.

I create a command and extract the data via this Xpath expression:

//response/forecast/txt_forecast/forecastdays/forecastday[2]/fcttext_metric

I create a custom sensor and then i display differents weather forecast in designer. it
works perfectly.

The
problem is that french words that contain accents (acute accent, circumflex,
etc) are not well reproduced. I get the result below:

In the XML, accents are OK.

I looked for a solution in the forum but I haven’t found. Does anyone has a clue?

Thank you in advance

Benoit

michal · September 18, 2016, 7:55am

You can try to add -Dfile.encoding=UTF-8 switch to the OpenRemote's Java command line.

Guy_levesque · September 19, 2016, 12:27am

Si cela fonctionne , j'aimerais bien savoir comment tu fais car je n,ai plus ce service , cela ne fonctionne plus avec yahoo. c'est plutôt comment fais-tu pour downloader en xml sur ton ordi!

Benoit_Mathieu · September 20, 2016, 8:45pm

Just follow this: http://www.openremote.org/display/docs/OpenRemote+2.0+How+To±+HTTP+Protocol
And for downloading the XML file, go to Availability of XML data in http://www.openremote.org/display/docs/OpenRemote+2.0+How+To±+Use+XML+Data+Sources#OpenRemote2.0HowTo-UseXMLDataSources-AvailabilityofXMLdata

Benoit_Mathieu · September 20, 2016, 11:23pm

Thanks Michal
What do you mean? I don’t know where to put this code. My command is a http request, no java code in

michal · September 21, 2016, 11:01am

Hi,

the procedure depends on how you start OpenRemote. If via openremote.sh then add
JAVA_OPTS="$JAVA_OPTS -Dfile.encoding=UTF-8"

to it. In case of openremote.bat

set JAVA_OPTS=%JAVA_OPTS% -Dfile.encoding=UTF-8

in case of professional eBox installation openremote is started by yajsw and then you need to add

wrapper.java.additional.8 = -Dfile.encoding=UTF-8

at the end of /opt/yajsw-stable-11.03/conf/wrapper.conf file.

You can check command line parameters on linux with:

ps ax|grep java

make sure that -Dfile.encoding=UTF-8 option is present.

Kind regards,

Michal

Benoit_Mathieu · September 21, 2016, 12:14pm

Where to put it? i place the code just in the first line of openremote.sh, then the "
ps ax|grep java " seems good. But no change in the display

michal · September 21, 2016, 12:42pm

Seems OK. Can you copy here exactly the URL and Xpath strings so I can try to reproduce the exact command myself?

Benoit_Mathieu · September 21, 2016, 1:35pm

The XML file is obtained by a crontab, every 15 min:
*/15 * * * * wget http://api.wunderground.com/api/API key/conditions/forecast/lang:FR/q/be/anderlues.xml -O /opt/openremote/webapps/private/weather_anderlues.xml > /opt/openremote/webapps/private/weather_cron.log 2>&1
This part work perfectly and the result looks like:

0.1 http://www.wunderground.com/weather/api/d/terms.html conditions forecast http://icons.wxug.com/graphics/wu2/logo_130x80.png Weather Underground http://www.wunderground.com Anderlues, Belgium Anderlues Belgium BX BE 00000 7 06449 50.40000153 4.26666689 186.00000000 Rue Benoit Fauconnier, Fontaine-l'ÃvÃªque, Rue Benoit Fauconnier, Fontaine-l'ÃvÃªque BE BE 50.410057 4.324953 465 ft IFONTAIN52 Last Updated on septembre 19, 18:07 CEST Mon, 19 Sep 2016 18:07:40 +0200 1474301260 Mon, 19 Sep 2016 18:13:33 +0200 1474301613 CEST Europe/Brussels +0200 Ciel dégagé 67.5 F (19.7 C) 67.5 19.7 53% From the ENE at 1.2 MPH Gusting to 4.3 MPH ENE 64 1.2 4.3 1.9 6.9 1020 30.12 0

    <dewpoint_string>50 F (10 C)</dewpoint_string>
    <dewpoint_f>50</dewpoint_f>
    <dewpoint_c>10</dewpoint_c>
   
   
    <heat_index_string>NA</heat_index_string>
    <heat_index_f>NA</heat_index_f>
    <heat_index_c>NA</heat_index_c>
   
   
    <windchill_string>NA</windchill_string>
    <windchill_f>NA</windchill_f>
    <windchill_c>NA</windchill_c>
   
    <feelslike_string>67.5 F (19.7 C)</feelslike_string>
    <feelslike_f>67.5</feelslike_f>
    <feelslike_c>19.7</feelslike_c>
    <visibility_mi>N/A</visibility_mi>
    <visibility_km>N/A</visibility_km>
    <solarradiation></solarradiation>
    <UV>-1</UV>
    <precip_1hr_string>-999.00 in ( 0 mm)</precip_1hr_string>
    <precip_1hr_in>-999.00</precip_1hr_in>
    <precip_1hr_metric> 0</precip_1hr_metric>
    <precip_today_string>0.00 in (0 mm)</precip_today_string>
    <precip_today_in>0.00</precip_today_in>
    <precip_today_metric>0</precip_today_metric>
   
   
   
    <icon>clear</icon>
    <icon_url>http://icons.wxug.com/i/c/k/clear.gif</icon_url>
    <forecast_url>http://www.wunderground.com/global/stations/06449.html</forecast_url>
   
    <history_url>http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=IFONTAIN52</history_url>
   
    <ob_url>http://www.wunderground.com/cgi-bin/findweather/getForecast?query=50.410057,4.324953</ob_url>
</current_observation>

   






   

<forecast>
    <txt_forecast>
        <date>17:40 CEST</date>
        <forecastdays>
           
           

       
            <forecastday>
                <period>0</period>
                <icon>partlycloudy</icon>
                <icon_url>http://icons.wxug.com/i/c/k/partlycloudy.gif</icon_url>
               
                <title>lundi</title>
                <fcttext><![CDATA[Partiellement nuageux. Minimales : 50 ºF.]]></fcttext>
                <fcttext_metric><![CDATA[Partiellement nuageux. Minimales : 10 ºC.]]></fcttext_metric>
                <pop>10</pop>
            </forecastday>

           
            <forecastday>
                <period>1</period>
                <icon>nt_partlycloudy</icon>
                <icon_url>http://icons.wxug.com/i/c/k/nt_partlycloudy.gif</icon_url>
               
                <title>lundi soir</title>
                <fcttext><![CDATA[Partiellement nuageux. Minimales : 50 ºF. Vents NE soufflant de 5 à 10 mi/h.]]></fcttext>
                <fcttext_metric><![CDATA[Partiellement nuageux. Minimales : 10 ºC. Vents NE soufflant de 10 à 15 km/h.]]></fcttext_metric>
                <pop>10</pop>
            </forecastday>
           
       
            <forecastday>
                <period>2</period>
                <icon>partlycloudy</icon>
                <icon_url>http://icons.wxug.com/i/c/k/partlycloudy.gif</icon_url>
               
                <title>mardi</title>
                <fcttext><![CDATA[Partiellement nuageux. Maximales : 66 ºF. Vents ENE soufflant de 5 à 10 mi/h.]]></fcttext>
                <fcttext_metric><![CDATA[Partiellement nuageux. Maximales : 19 ºC. Vents ENE soufflant de 10 à 15 km/h.]]></fcttext_metric>
                <pop>10</pop>
            </forecastday>

           
            <forecastday>
                <period>3</period>
                <icon>nt_partlycloudy</icon>
                <icon_url>http://icons.wxug.com/i/c/k/nt_partlycloudy.gif</icon_url>
               
                <title>mardi soir</title>
                <fcttext><![CDATA[Partiellement nuageux. Minimales : 49 ºF. Vents ENE et variables.]]></fcttext>
                <fcttext_metric><![CDATA[Partiellement nuageux. Minimales : 9 ºC. Vents ENE et variables.]]></fcttext_metric>
                <pop>10</pop>
            </forecastday>
           
       
            <forecastday>
                <period>4</period>
                <icon>partlycloudy</icon>
                <icon_url>http://icons.wxug.com/i/c/k/partlycloudy.gif</icon_url>
               
                <title>mercredi</title>
                <fcttext><![CDATA[Partiellement nuageux. Maximales : 69 ºF. Vents SE et variables.]]></fcttext>
                <fcttext_metric><![CDATA[Partiellement nuageux. Maximales : 21 ºC. Vents SE et variables.]]></fcttext_metric>
                <pop>10</pop>
            </forecastday>

           
            <forecastday>
                <period>5</period>
                <icon>nt_partlycloudy</icon>
                <icon_url>http://icons.wxug.com/i/c/k/nt_partlycloudy.gif</icon_url>
               
                <title>mercredi soir</title>
                <fcttext><![CDATA[Partiellement nuageux. Minimales : 51 ºF. Vents SSE et variables.]]></fcttext>
                <fcttext_metric><![CDATA[Partiellement nuageux. Minimales : 10 ºC. Vents SSE et variables.]]></fcttext_metric>
                <pop>0</pop>
            </forecastday>
           
       
            <forecastday>
                <period>6</period>
                <icon>mostlycloudy</icon>
                <icon_url>http://icons.wxug.com/i/c/k/mostlycloudy.gif</icon_url>
               
                <title>jeudi</title>
                <fcttext><![CDATA[Très nuageux. Maximales : 70 ºF. Vents SSO soufflant de 5 à 10 mi/h.]]></fcttext>
                <fcttext_metric><![CDATA[Très nuageux. Maximales : 21 ºC. Vents SSO soufflant de 10 à 15 km/h.]]></fcttext_metric>
                <pop>0</pop>
            </forecastday>

           
            <forecastday>
                <period>7</period>
                <icon>nt_partlycloudy</icon>
                <icon_url>http://icons.wxug.com/i/c/k/nt_partlycloudy.gif</icon_url>
               
                <title>jeudi soir</title>
                <fcttext><![CDATA[Partiellement nuageux. Minimales : 51 ºF. Vents SO et variables.]]></fcttext>
                <fcttext_metric><![CDATA[Partiellement nuageux. Minimales : 11 ºC. Vents SO et variables.]]></fcttext_metric>
                <pop>10</pop>
            </forecastday>
           
       
        </forecastdays>
    </txt_forecast>

    <simpleforecast>
        <forecastdays>
           
            <forecastday>
                <date>

1474304400
<pretty_short>19:00 CEST</pretty_short>
19:00 CEST le 19 septembre 2016
19
9
2016
262
19
00
0
1
septembre
<monthname_short>Sep</monthname_short>
<weekday_short>lun</weekday_short>
lundi
PM
<tz_short>CEST</tz_short>
<tz_long>Europe/Brussels</tz_long>

1

78
25

50
10

Partiellement nuageux

                <icon>partlycloudy</icon>
                <icon_url>http://icons.wxug.com/i/c/k/partlycloudy.gif</icon_url>
                <skyicon></skyicon>
                <pop>10</pop>
                <qpf_allday>
                    <in>0.00</in>
                    <mm>0</mm>
                </qpf_allday>
                <qpf_day>
                    <in></in>
                    <mm></mm>
                </qpf_day>
                <qpf_night>
                    <in>0.00</in>
                    <mm>0</mm>
                </qpf_night>
                <snow_allday>
                    <in>0.0</in>
                    <cm>0.0</cm>
                </snow_allday>
                <snow_day>
                    <in></in>
                    <cm></cm>
                </snow_day>
                <snow_night>
                    <in>0.0</in>
                    <cm>0.0</cm>
                </snow_night>
                <maxwind>
                    <mph>9</mph>
                    <kph>14</kph>
                    <dir></dir>
                    <degrees>0</degrees>
                </maxwind>
                <avewind>
                    <mph>1</mph>
                    <kph>1</kph>
                    <dir>ENE</dir>
                    <degrees>61</degrees>
                </avewind>
                <avehumidity>75</avehumidity>
                <maxhumidity>0</maxhumidity>
                <minhumidity>0</minhumidity>
            </forecastday>
           
            <forecastday>
                <date>

1474390800
<pretty_short>19:00 CEST</pretty_short>
19:00 CEST le 20 septembre 2016
20
9
2016
263
19
00
0
1
septembre
<monthname_short>Sep</monthname_short>
<weekday_short>mar</weekday_short>
mardi
PM
<tz_short>CEST</tz_short>
<tz_long>Europe/Brussels</tz_long>

2

66
19

49
9

Partiellement nuageux

                <icon>partlycloudy</icon>
                <icon_url>http://icons.wxug.com/i/c/k/partlycloudy.gif</icon_url>
                <skyicon></skyicon>
                <pop>10</pop>
                <qpf_allday>
                    <in>0.00</in>
                    <mm>0</mm>
                </qpf_allday>
                <qpf_day>
                    <in>0.00</in>
                    <mm>0</mm>
                </qpf_day>
                <qpf_night>
                    <in>0.00</in>
                    <mm>0</mm>
                </qpf_night>
                <snow_allday>
                    <in>0.0</in>
                    <cm>0.0</cm>
                </snow_allday>
                <snow_day>
                    <in>0.0</in>
                    <cm>0.0</cm>
                </snow_day>
                <snow_night>
                    <in>0.0</in>
                    <cm>0.0</cm>
                </snow_night>
                <maxwind>
                    <mph>10</mph>
                    <kph>16</kph>
                    <dir>ENE</dir>
                    <degrees>58</degrees>
                </maxwind>
                <avewind>
                    <mph>6</mph>
                    <kph>10</kph>
                    <dir>ENE</dir>
                    <degrees>58</degrees>
                </avewind>
                <avehumidity>64</avehumidity>
                <maxhumidity>0</maxhumidity>
                <minhumidity>0</minhumidity>
            </forecastday>
           
            <forecastday>
                <date>

1474477200
<pretty_short>19:00 CEST</pretty_short>
19:00 CEST le 21 septembre 2016
21
9
2016
264
19
00
0
1
septembre
<monthname_short>Sep</monthname_short>
<weekday_short>mer</weekday_short>
mercredi
PM
<tz_short>CEST</tz_short>
<tz_long>Europe/Brussels</tz_long>

3

69
21

51
11

Partiellement nuageux

                <icon>partlycloudy</icon>
                <icon_url>http://icons.wxug.com/i/c/k/partlycloudy.gif</icon_url>
                <skyicon></skyicon>
                <pop>10</pop>
                <qpf_allday>
                    <in>0.00</in>
                    <mm>0</mm>
                </qpf_allday>
                <qpf_day>
                    <in>0.00</in>
                    <mm>0</mm>
                </qpf_day>
                <qpf_night>
                    <in>0.00</in>
                    <mm>0</mm>
                </qpf_night>
                <snow_allday>
                    <in>0.0</in>
                    <cm>0.0</cm>
                </snow_allday>
                <snow_day>
                    <in>0.0</in>
                    <cm>0.0</cm>
                </snow_day>
                <snow_night>
                    <in>0.0</in>
                    <cm>0.0</cm>
                </snow_night>
                <maxwind>
                    <mph>5</mph>
                    <kph>8</kph>
                    <dir>SE</dir>
                    <degrees>132</degrees>
                </maxwind>
                <avewind>
                    <mph>4</mph>
                    <kph>6</kph>
                    <dir>SE</dir>
                    <degrees>132</degrees>
                </avewind>
                <avehumidity>68</avehumidity>
                <maxhumidity>0</maxhumidity>
                <minhumidity>0</minhumidity>
            </forecastday>
           
            <forecastday>
                <date>

1474563600
<pretty_short>19:00 CEST</pretty_short>
19:00 CEST le 22 septembre 2016
22
9
2016
265
19
00
0
1
septembre
<monthname_short>Sep</monthname_short>
<weekday_short>jeu</weekday_short>
jeudi
PM
<tz_short>CEST</tz_short>
<tz_long>Europe/Brussels</tz_long>

4

70
21

51
11

Nuageux

                <icon>mostlycloudy</icon>
                <icon_url>http://icons.wxug.com/i/c/k/mostlycloudy.gif</icon_url>
                <skyicon></skyicon>
                <pop>0</pop>
                <qpf_allday>
                    <in>0.00</in>
                    <mm>0</mm>
                </qpf_allday>
                <qpf_day>
                    <in>0.00</in>
                    <mm>0</mm>
                </qpf_day>
                <qpf_night>
                    <in>0.00</in>
                    <mm>0</mm>
                </qpf_night>
                <snow_allday>
                    <in>0.0</in>
                    <cm>0.0</cm>
                </snow_allday>
                <snow_day>
                    <in>0.0</in>
                    <cm>0.0</cm>
                </snow_day>
                <snow_night>
                    <in>0.0</in>
                    <cm>0.0</cm>
                </snow_night>
                <maxwind>
                    <mph>10</mph>
                    <kph>16</kph>
                    <dir>SSO</dir>
                    <degrees>193</degrees>
                </maxwind>
                <avewind>
                    <mph>7</mph>
                    <kph>11</kph>
                    <dir>SSO</dir>
                    <degrees>193</degrees>
                </avewind>
                <avehumidity>59</avehumidity>
                <maxhumidity>0</maxhumidity>
                <minhumidity>0</minhumidity>
            </forecastday>
           
        </forecastdays>
    </simpleforecast>
</forecast>

I try to display the " <fcttext_metric><![CDATA"

In Open remote designer, the command is:
URL : http://local ip:8688/private/weather_anderlues.xml
method: GET
Xpath expression: //response/forecast/txt_forecast/forecastdays/forecastday[2]/fcttext_metric
polling interval: 1m

Thanks for your help

Guy_levesque · September 23, 2016, 4:12pm

thank you it works, now I have the same problem as you.

Jerome_Jerome · October 4, 2017, 7:30pm

Hello Benoit,

Did you solve your problem ? I have exactly the same problem as in my conf :

Raspberry PI 3 Raspbian Stretch

Z-stick Gen 5

Openremote controller 2.6

Regards,

Jerome

Jerome_Jerome · October 5, 2017, 12:35pm

Hello All,

As said in my previous post, I have the same problem as Benoit.
Before adding thispost I have checked my xml file and already try to add the line JAVA_OPTS="$JAVA_OPTS -Dfile.
encoding=UTF-8" in my openremote.sh fil but it hasn t changed anything.

I have also tried to use the “TRANSLATE” function in my XPATH feld in order to change all accents : éèàç into eeac but it as also not changed anything.

So it looks as if the problem of accent happens when the GET is done and so before the XPATH field is interpreted.

Here is a screen shot of the interface I have developped for my Iphone 6 and it is a pity to have to this accent problem.

If someone could help it would be very nice.

Thanks for your help,

Jérôme (with the accents )

Jerome_Jerome · October 5, 2017, 7:33pm

Hello All,
I have found where the problem was and here is the solution (could be usefull for others )
In fact, even if the xml files downloaded via an sh script using the wget command are coded in UTF-8 ( you can see it with : file -i YouFile )
I have had to recode them in LATIN1 in order to get the accents displayed correctly.
Doing it is very simple, just add an inconv command into the script.
So the script looks like :
#/!bin/bash

Current Weather => get the xml file from openweathermap web site

wget -q -N “http://api.openweathermap.org/data/2.5/weather?id=2992132&mode=xml&units=metric&appid=[My Openweather map ID]&lang=fr” --output-document=/home/pi/openremote/webapps/private/weather_actuel.xml

Conversion to LATIN1 pour les accents

iconv -f UTF-8 -t LATIN1 /home/pi/openremote/webapps/private/weather_actuel.xml -o /home/pi/openremote/webapps/private/weather_actuel.xml

Now, accents are displayed well