Surfing Bits

MetaHome: Interface for Contolling Home Devices

dexity — Sun, 09 Oct 2011 19:00:40 +0000

Code: https://bitbucket.org/dexity/metahome

Controlling home devices from a single hand held device is a dream that will soon become real. Home devices are getting smarter and include more functionality than devices we have now. I developed a simple prototype of Android application that can manage virtual devices. The idea can be userful both for Home Automation to control devices as well as for Smart Maps to display detailed information about the objects on the map such as airports or shopping centers.

Application Description

The application manages predefined items (“Restroom”, “Telephone”, “Elevator”, “ATM”) by adding or removing them from display. The items can represent any object on the map: controllable home device or non-controllable object on the map. The application uses 3 independent data sources: Memory, Local Database and Web Service. I implemented different data sources to explore user experience. User can select any of the data sources from the Menu. For Web Service I wrote a simple PHP script and put it on remote web server which handles incoming requests from the application. Adding item to the map is as simple as clicking on the dashed square and selecting the item.

Layout and Drawing

The layout consists of three main views: MapView, TextView and ListView.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

xmlns:android="http://schemas.android.com/apk/res/android"
android:orientation="vertical"
android:layout_width="match_parent"
android:layout_height="match_parent">

android:id="@+id/map_view"
android:layout_width="wrap_content"
android:layout_height="240dip"
android:layout_margin="10dip"/>

android:id="@+id/status"
android:layout_width="wrap_content"
android:layout_height="23dip"
android:textSize="14dip"/>

android:id="@android:id/list"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:drawSelectorOnTop="false"/>

>

Both MapView and ListView are different views of the same data model described in the next section. TextView displays the status of selected data source. When the screen is rotated the application will use a different layout: layout-land/main.xml

MapView performs custom drawing of items depending if an item is located in the square or not.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

@Override
protected void onDraw(Canvas canvas)
{
super.onDraw(canvas);

float[] outerR = new float[] { 10, 10, 10, 10, 10, 10, 10, 10 };

mShapeDrawable = new ShapeDrawable(new RoundRectShape(outerR, null, null));
mShapeDrawable.getPaint().setColor(0xcccccccc);
mShapeDrawable.setBounds(0, 0, MAPSIZE, MAPSIZE);
mShapeDrawable.draw(canvas);

for (int i = 0; i < ldata.size(); i++)
{
if (ldata.isEmpty(i))
{
drawPlace(canvas, i);
}
else {
drawIcon(canvas, i);
}
}
}

Data Model and Interface

Data model of the application is very simple. It is described by a list of size 4. The index of the list represents cell index of flattened table: from left to right, top to bottom. The value of the list represents index of item which is placed in the cell. Empty cell has value -1.

Items indices:
0 – Restroom
1 – Telephone
2 – Elevator
3 – ATM

For example:

1	["0","-1","-1","2"]

means that item “Restroom” (item index 0) is in cell (0, 0) (cell index 0) and “Elevator” (item index 2) is in cell (1, 1) (cell index 3).

To make operations on the cells independent on the data source I implemented a general interface. All manipulation with the data storage is performed through this interface. If you want to add a new data storage you need to add also implementation of the interface.

1
2
3
4
5
6
7
8

public interface IMapData {
abstract void set(int index, int value);
abstract int get(int index);
abstract int[] getAll();
abstract void remove(int index);
abstract boolean isEmpty(int index);
abstract int size();
}

Cells list is then translated to ImageText class which is used by views to display image and corresponding label.

1
2
3
4
5
6
7
8
9
10

class ImageText
{
String text;
Bitmap icon;
public ImageText(Bitmap icon, String text)
{
this.icon = icon;
this.text = text;
}
}

Data Sources

As mentioned before, I implemented 3 independent data sources each of which can be selected from the application:

Memory
Local Database
Web Service

Local Database is a simple SQLite database which resides on Android file system and has the following database schema:

1
2
3
4
5
6
7
8
9

$ sqlite3 cells.db
sqlite> .schema
CREATE TABLE android_metadata (locale TEXT);
CREATE TABLE cells (_id INTEGER PRIMARY KEY,cid INTEGER,cell INTEGER);
sqlite> select * from cells;
1|0|0
2|1|2
3|2|1
4|3|-1

I will show here the examples of implementation set() method for each of these data sources:

Memory

1
2
3
4
5
6
7
8
9

class MemoryMapData implements IMapData
{
private int[] cells = new int[MetaHome.NUM_ITEMS];

public void set(int index, int value) {
cells[index] = value;
}
...
}

Local Database

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

class DatabaseMapData implements IMapData
{
private DatabaseHelper mOpenHelper;

private static final String DBNAME = "cells.db";
private static final int DBVERSION = 3;

public void set(int index, int value) {

SQLiteDatabase db = mOpenHelper.getWritableDatabase();
ContentValues values = new ContentValues();
values.put(Cells.COLUMN_CELL, value);
db.update(Cells.TABLE_NAME, values, Cells.COLUMN_ID+"=?", new String[] {Long.toString(index)});
}
...
}

Web Service

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

class WebMapData implements IMapData
{

private static final String ENDPOINT = "http://surfingbits.com/metahome/index.php";
private static final String KEY = "access_key";

public void set(int index, int value) {
HashMap<String, String> params = new HashMap<String, String>();
params.put("action", "set");
params.put("cell", Long.toString(index));
params.put("value", Long.toString(value));
JSONObject js = getContentObject(params);

boolean status;
try {
status = js.getBoolean("status");
Log.i("Web", "Set value cell["+index+"] = "+value+": "+status);
} catch (Exception e) {
Log.e("WebError", e.toString());
}
}
...
}

Web Service Interface and Implementation

For Web Service I implemented a simple PHP script which handles incoming requests from the application. In the backend the script also uses SQLite database with the same database schema as Android is using for the Local Database source.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166

// Copyright 2011 Alex Dementsov

$key = "access_key";
$actions = array("get", "getall", "remove", "set", "size", "empty");

class Handler
{
private $db;

public function __construct(){
$this->open_db();
}

function run()
{
global $key, $actions;

if (!(isset($_GET["key"]) && $_GET["key"] == $key) ||
!(isset($_GET["action"]) && in_array($_GET["action"], $actions)))
{
echo "Wrong key or action";
return;
}

switch($_GET["action"])
{
case "get":
return $this->get();
case "getall":
return $this->get_all();
case "remove":
return $this->remove();
case "set":
return $this->set();
case "size":
return $this->size();
case "empty":
return $this->isempty();
}
}
function get()
{
if (!isset($_GET["cell"]))
return $this->status_fail();

$row = $this->getValue();
if (!$row)
return $this->status_fail();

return json_encode(array("cell" => $row["cid"], "value" => $row["cell"]));
}

function get_all()
{
$array = array(-1, -1, -1, -1);
$rows = $this->getValues();
if (!$rows)
return $this->status_fail();

foreach ($rows as $i => $value)
{
$idx = $value["cid"];
if ($idx >= 0 && $idx < 4)
$array[$idx] = $value["cell"];
}
return json_encode(array("cells" => $array));
}

function remove()
{
if (!isset($_GET["cell"]))
return $this->status_fail();

return $this->setValue(-1);
}

function set()
{
if (!isset($_GET["cell"]) or !isset($_GET["value"]))
return $this->status_fail();

return $this->setValue($_GET["value"]);
}

private function update($arr)
{
// Executes query and returns status
$query = "update cells set cell=? where cid=?";
$q = $this->db->prepare($query);
$q->execute($arr) or die(print_r($this->db->errorInfo(), true));
return $q;
}

private function getValue()
{
// Returns value of a cell
$result = $this->db->query("select * from cells where cid=".$_GET["cell"]) or die($this->status_fail());
return $result->fetch();
}

private function getValues()
{
// Returns value of a cell
$result = $this->db->query("select * from cells limit 4") or die($this->status_fail());
return $result->fetchAll();
}

private function setValue($value)
{
$arr = array($value, $_GET["cell"]);
$q = $this->update($arr);
if (!$q)
return $this->status_fail();

return $this->status_ok();
}

function size()
{
return json_encode(array("size" => 4));
}

function isempty()
{
$row = $this->getValue();
if (!$row)
return $this->status_fail();

if ($row["cell"] == -1)
return json_encode(array("empty" => True));

return json_encode(array("empty" => False));
}

private function status_fail()
{
return $this->status("fail");
}

private function status_ok()
{
return $this->status("ok");
}

private function status($status)
{
return json_encode(array("status" => $status));
}

private function open_db()
{
try {
$this->db = new PDO('sqlite:db/cells.db');
}
catch(Exception $e) {
die($this->status_fail());
}
}
}

$handler = new Handler();
echo $handler->run();
?>

Here is the interface for the Web Service:

Get All Cells

1
2
3
4
5
6
7

Example Request:
http://HOSTNAME/metaservice/index.php?action=getall&key=access_key

Example Response:
{
“cells”: [“2”, “3”, “-1”, “-1”]
}

Get Cell

1
2
3
4
5
6
7
8

Example Request:
http://HOSTNAME/metaservice/index.php?action=get&cell=3&key=access_key

Example Response:
{
“cell”: 3,
“value”: 2
}

Remove Cell

1
2
3
4
5
6
7

Example Request:
http://HOSTNAME/metaservice/index.php?action=remove&cell=3&key=access_key

Example Response:
{
“status”: “ok”
}

Set Cell

1
2
3
4
5
6
7

Example Request:
http://HOSTNAME/metaservice/index.php?action=set&cell=3&value=2&key=access_key

Example Response:
{
“status”: “ok”
}

Get Cells Size

1
2
3
4
5
6
7

Example Request:
http://HOSTNAME/metaservice/index.php?action=size&key=access_key

Example Response:
{
“size”: 4
}

Is Empty

1
2
3
4
5
6
7

Example Request:
http://HOSTNAME/metaservice/index.php?action=empty&cell=3&key=access_key

Example Response:
{
“empty”: true
}

If something goes wrong the response returns error:

1
2
3

{
“status”: “fail”
}

Caching Mechanism and Other Trick

Using Web Service data source without caching makes user experience not very pleasant. I implemented some caching mechanism to make experience better. Remote calls are made to Web Service only when it is absolutely necessary, for example to change the state of the cells. Other requests are made locally from the stored data attribute ldata.

1
2
3
4
5

public class MapView extends View {

private IMapData ldata; // Local storage for the data (cache)
...
}

This attribute is used for all three data sources even when it somewhat duplicates storage (as in Memory data source). This simple caching works surprisingly well for Web Service data source.

Other issue I had with the application is the screen rotation. When you rotate the mobile phone all the local objects are gone because Android restarts the running Activity unless you take some measures to keep the state. Here is the reference to Android documentation which discusses this issue: http://developer.android.com/guide/topics/resources/runtime-changes.html

To do the trick I implemented onSaveInstanceState() and onRestoreInstanceState(Parcelable state).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

public class MapView extends View {

@Override
protected Parcelable onSaveInstanceState() {
Bundle b = new Bundle();

Parcelable s = super.onSaveInstanceState();
b.putParcelable("map_super_state", s);
b.putIntArray("map_cells", dumpCells(ldata));
b.putIntArray("memory_cells", dumpCells(memory));
return b;
}

@Override
protected void onRestoreInstanceState(Parcelable state)
{
Bundle b = (Bundle) state;
Parcelable superState = b.getParcelable("map_super_state");
loadCells(ldata, b, "map_cells");
loadCells(memory, b, "memory_cells");

super.onRestoreInstanceState(superState);
}
...
}

Real Example

To show the possible application of the idea I present here the map of the Terminal 4 at Los Angeles International Airport (LAX). It looks a bit more complicated than what I showed you before .

Analysis of TuneIn Radio Stations

dexity — Fri, 08 Jul 2011 16:38:35 +0000

Code: https://bitbucket.org/dexity/surfsnippets/src/1b6a53d8c3e3/stationdigger/
Output: station_lang.json (6.9 Mb), station_genre.json (6.4 Mb)

Recently I was interested in TuneIn (http://tunein.com) radio service which provides access to radio stations around the globe with a pretty simple REST API: http://inside.radiotime.com/developers/api/opml

I wrote a script stationdigger.py which traverses tree of links and collects information about the stations. In this post I will uncover statistics behind TuneIn radio stations and show how to use the script to do some cool thinks with the TuneIn API.

TuneIn radio API provides both OPML (XML-like) and JSON formats. You don’t have to have any credentials to use this service. But if you start to use the service frequently you will end up getting 403 Forbidden error. Let’s try to make the first request:

1	http://opml.radiotime.com/Browse.ashx

Your response will be in default OPML format:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

version="1">
>
>TuneIn>
>200>
>
>
type="link" text="Local Radio" URL="http://opml.radiotime.com/Browse.ashx?c=local" key="local"/>
type="link" text="Music" URL="http://opml.radiotime.com/Browse.ashx?c=music" key="music"/>
type="link" text="Talk" URL="http://opml.radiotime.com/Browse.ashx?c=talk" key="talk"/>
type="link" text="Sports" URL="http://opml.radiotime.com/Browse.ashx?c=sports" key="sports"/>
type="link" text="By Location" URL="http://opml.radiotime.com/Browse.ashx?id=r0" key="location"/>
type="link" text="By Language" URL="http://opml.radiotime.com/Browse.ashx?c=lang" key="language"/>
type="link" text="Podcasts" URL="http://opml.radiotime.com/Browse.ashx?c=podcast" key="podcast"/>
>
>

This is where the story begins. Traversing tree of links with type=”link” you can reach links with type=”audio” and collect information about the stations. For example, one branch of the link tree can look like:

1
2
3

http://opml.radiotime.com/Browse.ashx
+-- <outline type="link" text="Local Radio" URL="http://opml.radiotime.com/Browse.ashx?c=local" key="local"/>
+-- <outline type="audio" text="San Diego Police Scanners: 2 (Police)" URL="http://opml.radiotime.com/Tune.ashx?id=s103434" bitrate="32" reliability="86" guide_id="s103434" subtext="English" genre_id="g2741" formats="mp3" item="station" image="http://radiotime-logos.s3.amazonaws.com/s103434q.png" now_playing_id="s103434" preset_id="s103434"/>

My script stationdigger.py just makes this task automatic and more error prone.

Results Format

I collected statistics for radio stations arranged by language and genre. The results are stored in files station_lang.json (6.9 Mb) and station_genre.json (6.4 Mb) in JSON format for stations by language and genre correspondingly.

Stations by language has the format:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

{<language id>:
{"name": <language name>,
"stations":
{<station id>:
{"name": <station name: {text}>,
"description": <station description: {subtext}>,
"url": <station url: {URL}>,
"image": <station image: {image}> ,
"genre": <genre id: {genre_id}>
},
# other stations
}
}
# other languages
}

For example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

{"l191":
{"name": "Afrikaans",
"stations":
{"s101967":
{"name": "Radio Namakwaland (Vredendal, South Africa)",
"description": "Afrikaans"
"url": "http://opml.radiotime.com/Tune.ashx?id=s101967&filter=l191",
"image": "http://radiotime-logos.s3.amazonaws.com/s101967q.png",
"genre": "g133",
}
...
}
}
...
}

Stations by genre has similar format:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

{<genre id>:
{"name": <genre name>,
"stations":
{<station id>:
{"name": <station name>,
"description": <station description>,
"url": <station url>,
"image": <station image>,
"lang": <language id>
},
# other stations
}
}
# other genres
}

For example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

{"g2738":
{"name": "Consumer & Technology",
"stations":
{"s124197":
{"name": "Radio FrequencyCast UK Tech (UK)",
"description": "Technology radio show from..."
"url": "http://opml.radiotime.com/Tune.ashx?id=s124197&filter=l1",
"image": "http://radiotime-logos.s3.amazonaws.com/s124197q.png",
"lang": "l1",
}
...
}
...
}

Parameter url holds the link to the station stream which you save on your machine and open content of the file with some media player.

For example, open the link in the browser:

1	http://opml.radiotime.com/Tune.ashx?id=s124197&filter=l1

and in the Tune.ashx file you will find station stream which looks something like:

1	http://lush.wavestreamer.com:3680/

Open this link, for example, in Banshee Media Player (Media/Open Location) available in Ubuntu as a default media player and enjoy the station

Code Analysis

As I mentioned before, stationdigger.py code traverses tree of links and collects information about the stations using DFS algorithm. At first I thought that it will be pretty straightforward to implement the algorithm, but later I realized that I have to handle several issues specific to TuneIn service to properly do it.

TuneIn radio service significantly limits number of calls without partnerId passed (http://inside.radiotime.com/developers/api/opml/security-model#limits). It also has limits for calls even with partnerId but the limit is larger than without partnerId. Once the limit is reached you will start getting 403 Forbidden error for a day. Because of the large number of calls needed to collect the data it is good to have partnerId or you can wait for a long time. To collect stations using partnerId it took me about 4-5 days. It is also recommended to pass the serial parameter which can be pretty much anything.

The core of the algorithm is method traverse():

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

def traverse(self, url):
"Traverses tree of links and populates stations using DFS algorithm"
if not url or (self.numTotal and self.numTotal<self.num):
return

items = self.make_call(url=url)
(audios, links) = self._separate(items)
for item in audios: # check audio links first: playable content
if item.has_key("guide_id") and \
not self._stations.has_key(item.get("guide_id")): # ignore duplicates
self._stations[item.get("guide_id")] = self.to_station(item) # set station
self.num += 1
if self.numTotal and self.numTotal<self.num:
return

for item in links: # check links
self.traverse(item.get("URL"))

In this method it makes call to specified url and returns list of items in the tag. Then it separates links and audio links. Links can later be traversed down the link tree. Audio links are what we are after: they are used to collect information about the station.

The code supports the following features.

Can dig stations both for one and all languages

Because digging stations is a relatively long process, when something happens during the data collection you need a way to restart from some intermediate step. For this purpose the program stores stations in junks: one language in a separate file in a format:

lang.json

These files are stored in stations directory and then later merged with merge_langs.py script into a single file: station_lang.json. Besides collecting stations for all languages at once you can collect stations just for one language by passing a language tuple.

1
2
3
4
5
6
7
8

def dig_stations_lang(self, filename=None, lang_tuple=None):
"Diggs for stations with language filters"
if lang_tuple:
filters = [lang_tuple] # one language
else:
langjs = self.get_lang() # all languages
filters = self.sorted_tuple(langjs)
self._dig_stations(filters, {"c": "lang"}, filename)

Can dig specified number of stations

You don’t need to collect all stations. Setting limit on total number of stations (self.numTotal) will collect at most this number of stations for the filter passed. See implementation of method traverse(self, url). For example, if filter has two languages then stations will get populated starting from the first language.

Digs only unique stations

To make sure that stations are not duplicated for the same language the program checks if the station has been added before:

1
2
3
4
5
6

def traverse(self, url):
...
if item.has_key("guide_id") and \
not self._stations.has_key(item.get("guide_id")): # ignore duplicates
self._stations[item.get("guide_id")] = self.to_station(item) # set station
...

Handles “403 Forbidden” error

This error was a frequent companion during collecting data. This just means that limit of using the TuneIn service has been reached. It not handled properly you will end up either with non-complete stations or no stations at all. I did some experiments with this error and figured out that your program can be rejected either for one day or for a few minutes (not sure how this happens). So I developed some mechanism that handles this error which can be reduced to just one word: persistence! When program receives 403 Forbidden error it waits for some time and then tries to perform the same call again until it receives result other than this error.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

def make_call(self, url):
"Makes call and returns response in json format"
if not url:
return None
nurl = self._set_format(url)
if nurl in self._urls: # Avoid circular reference
return None
result = -1 # to enter the loop

while (result == -1):
try:
req = urllib2.Request(nurl, headers=self._headers())
response = urllib2.urlopen(req)
page = self._decompress(response.read()) # decompress
pdict = json.loads(page)
result = pdict.get("body") # interested in "body" section only: should be list
self._urls.add(nurl) # URL is processed
except urllib2.HTTPError, e:
if e.code == 403:
result = -1
else:
result = None
except:
result = None

if result == -1:
print "Waiting %s --> %s" % (self.num, nurl)
time.sleep(WAIT_TIME)
return result

Avoids circular reference

From experience I found out that some links might have circular reference: referring to itself in one or several hops. Once the program enters the circular reference it can get stuck forever! Simple check can save you out of trouble:

1
2
3
4
5

def make_call(self, url):
...
if nurl in self._urls: # Avoid circular reference
return None
...

Response compression

To optimize the stream of responses I added header to request encoded content supported by most of the popular web servers and decompress it on the program’s side:

1	Accept-Encoding: gzip

Here is the method which performs decompression:

1
2
3
4
5

def _decompress(self, s):
"Decompresses string"
sio = StringIO(s)
file = GzipFile(fileobj=sio)
return file.read()

Handles embedded links

Another issue I had related to TuneIn responses is embedded tags which looked like in xml format:

1
2
3

text="Stations" key="stations">
type="audio" text="Radio Pretoria (Kleinfontein)" URL="http://opml.radiotime.com/Tune.ashx?id=s10616&partnerId=ezxqdS53&serial=123&filter=l191" bitrate="32" reliability="96" guide_id="s10616" subtext="Tienie se musiek" genre_id="g255" formats="wma" show_id="p149910" item="station" image="http://radiotime-logos.s3.amazonaws.com/s10616q.png" current_track="Tienie se musiek" now_playing_id="s10616" preset_id="s10616"/>
>

To handle this issue I traverse also these subtrees:

1
2
3
4
5
6
7
8
9
10
11
12

def _traverse_outline(self, items, audios, links):
"Traverses children outline elements and populate audios and links lists"
if not items: # No items
return

for item in items:
if item.has_key("children") and item.get("children"):
self._traverse_outline(item.get("children"), audios, links)
elif item.get("type") == "audio":
audios.append(item)
elif item.get("type") == "link":
links.append(item)

Mimics headers of Chrome browser

For some reason passing browser headers with the call doesn’t reject calls as fast as without the headers. So I decided to explore Chrome browser headers and pretend that the calls are made with the browser. Here is the example of the headers presented as a Python dictionary:

1
2
3
4
5
6
7
8
9

headers = {
"Connection": "keep-alive",
"Cache-Control": "max-age=0",
"User-Agent": "Mozilla/5.0 (X11; Linux i686) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.71 Safari/534.24",
"Accept": "application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5",
"Accept-Encoding": "gzip",
"Accept-Language": "en-US,en;q=0.8",
"Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.3"
}

Using Script

Before you start running program it is recommended to set config.txt file first:

1
2
3

[general]
partnerId =
serial =

To collect stations for all languages you need to iterate over sorted languages:

1
2
3
4
5

sd = StationDigger()
langjs = json.load(open("generated/lang.json"))
lang = sd.sorted_tuple(langjs)
for n in range(len(lang)):
sd.dig_stations_lang("stations/lang%s.json" % n, lang_tuple=lang[n])

so the session for calls without partnerId passed looks like:

1
2
3

Finished: Aboriginal, total: 2
Finished: Afrikaans, total: 12
Waiting 15 --> http://opml.radiotime.com/Browse.ashx?id=g373&filter=l210&render=json

If you want to just dig stations for n-th language then the script will do the work:

1
2
3
4
5

sd = StationDigger()
langjs = json.load(open("generated/lang.json"))
lang = sd.sorted_tuple(langjs)
lang_tuple = lang[n]
sd.dig_stations_lang("stations/lang%s.json" % n, lang_tuple=lang_tuple)

Other Useful Scripts

Besides implementing stationdigger.py I also implemented some useful scripts performing some operations on generated files.

merge_langs.py

This script merges files with stations by a single language into one file: station_lang.json. You need to have language files already generated from stations collections. The file structure should look like:

1
2
3
4

stations/
lang0.json
lang10.json
...

lang2genre.py

It turns out that the stations by language can be converted into stations by genre because most of the stations have also genre field. This way one can collect statistics of station distribution by genre. lang2genre.py script converts station_lang.json into station_genre.json.

status_generator.py

This script uses station_lang.json and station_genre.json to generate cumulative statistics on languages and genres. The result is later used to draw plots.

draw_plots.py

Script for generating pie and histogram plots for languages and genres.

Plotting Results

This is the final step in our analysis. We take data generated by status_generator.py script and use draw_plots.py to create plots for languages and genres. To draw plots I used matplotlib – an advanced plotting Python library extensively used by scientists. I used pie and histogram plot types to represent data in two different ways. I wasn’t happy with the default color schema so I wrote function which generates html-type colors from HSV colors.

Here are the results I obtained:

Results Analysis

It is no surprise that TuneIn radio service is a US based company so probably they are more biased to English language. English speaking stations comprise 42.7% of all provided stations. Next popular language, Spanish and Portuguese, have about 10% each.

Genre distribution is not that narrow as for languages. Majority of stations feature pop genre (Top 40-Pop: 15.4 % and Adult Contemporary: 6.0%) which is also predictable. Detailed statistics can be found in status_lang_sort.txt and status_genre_sort.txt

Sea World Tiles

dexity — Tue, 24 May 2011 06:21:09 +0000

After hanging out you with my friends from Oregon to the SeaWorld in San Diego, CA I decided that it is might be cool to develop a simple Android application which keeps our memory about the adventure parks.

So here is the app called Sea World Tiles. Feel free to download it from Android Market and play around. In this app you need to compose the picture of a sea creature from left and right sides by clicking the appropriate buttons.

The source code is available in the repository surfsnippets in directory shamu. Here is the short description of source code to show how simple it is to write Android applications.

The layout in Android applications is described in .xml file, main.xml and includes TableLayout with two TableRow. The upper TableRow displays two tiles of ImageView and the bottom TableRow keeps left and right buttons:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

version="1.0" encoding="utf-8"?>
xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="fill_parent"
android:layout_height="fill_parent"
android:orientation="vertical"
android:id="@+id/main_layout">
android:id="@+id/tableLayout1" android:layout_width="match_parent" android:layout_height="wrap_content">
android:id="@+id/tableRow1" android:layout_width="wrap_content" android:layout_height="wrap_content">
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:src="@drawable/seaworld_l"
android:id="@+id/picture_l">>
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:src="@drawable/seaworld_r"
android:id="@+id/picture_r">>
>
android:id="@+id/tableRow2" android:layout_width="wrap_content" android:layout_height="wrap_content">
android:text="Left"
android:id="@+id/button_left"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_weight="0.5"
android:layout_margin="5dip">>
android:text="Right"
android:id="@+id/button_right"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_weight="0.5"
android:layout_margin="5dip">>
>
>
>

Layout specifies some important parameters such as widget id: @android:id=”+id/button_left”, sizes: android:layout_weight=”0.5″ or relation to container widget: android:layout_margin=”5dip”. Some of these parameters can be used in program.

The program is consists of just one Shamu.java:

package com.surfingbits.shamu;

import java.util.HashMap;

import android.app.Activity;
import android.os.Bundle;
import android.view.View;
import android.widget.Button;
import android.widget.ImageView;

import java.util.Random;

public class Shamu extends Activity {
/** Called when the activity is first created. */

HashMap<Integer, String> num2pic;
private static Random randNum;
public static int randl, randr;

@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);

initValues();

Button btnLeft = (Button) findViewById(R.id.button_left); // Left button
btnLeft.setOnClickListener(new Button.OnClickListener() {
public void onClick(View v) {
Shamu.this.replaceImage("l");
}
});
Button btnRight = (Button) findViewById(R.id.button_right); // Right button
btnRight.setOnClickListener(new Button.OnClickListener() {
public void onClick(View v) {
Shamu.this.replaceImage("r");
}
});
}

public void initValues()
{
randNum = new Random();
randl = 0;
randr = 0;
num2pic = new HashMap<Integer, String>();
num2pic.put(new Integer(0), "shamu");
num2pic.put(new Integer(1), "dolphin");
num2pic.put(new Integer(2), "seastar");
num2pic.put(new Integer(3), "rayfish");
}

public int getRandNum(String side)
{
// Two static variables randl and randr are set by corresponding button
try {
int temp = Shamu.class.getField("rand"+side).getInt(null); // randr or randl
int tempB = temp;
while (tempB == temp)
{ // try until you get different int
temp = randNum.nextInt(num2pic.size());
}
Shamu.class.getField("rand"+side).setInt(Shamu.class, temp);
return temp;
} catch (NoSuchFieldException e) {
return 0;
} catch (SecurityException e) {
return 0;
} catch (IllegalAccessException e) {
return 0;
}
}

public int layoutId(String base, Class cls, String side)
{
// Trying to use reflection getField (similar to Python getattr) to get field value
int id;
// Check if side is "l" or "r"
try {
id = cls.getField(base+"_"+side).getInt(null); // Example: R.id.picture_l
} catch (NoSuchFieldException e) {
id = 0;
} catch (SecurityException e) {
id = 0;
} catch (IllegalAccessException e) {
id = 0;
}
return id;
}

public void replaceImage(String side)
{
ImageView pic = (ImageView) findViewById(layoutId("picture", R.id.class, side));
pic.setImageResource(layoutId(num2pic.get(getRandNum(side)), R.drawable.class, side));
}
}

When Android application is started the onCreate() method is called.
First, it loads the main layout:

1	setContentView(R.layout.main);

initializes attributes and and then creates button widget specified by id R.id.button_left and defines the OnClickListener():

1
2
3
4
5
6

Button btnLeft = (Button) findViewById(R.id.button_left); // Left button
btnLeft.setOnClickListener(new Button.OnClickListener() {
public void onClick(View v) {
Shamu.this.replaceImage("l");
}
});

When the user clicks the button replaceImage() method will be called:

1
2
3
4
5

public void replaceImage(String side)
{
ImageView pic = (ImageView) findViewById(layoutId("picture", R.id.class, side));
pic.setImageResource(layoutId(num2pic.get(getRandNum(side)), R.drawable.class, side));
}

This method takes the ImageView by id and replaces it with another image by randomly generating number and looking up the appropriate id in the HashMap for the next picture.

To make the source code more compact I used Java Reflection: Reflection (computer programming) that can refer to class attributes or methods by name string. This approach is similar to Python functions getattr() or setattr().

Hello world!

dexity — Sun, 20 Feb 2011 06:46:42 +0000

In this blog I will explore cool technologies by doing some experiments on computer systems, small and large, and writing code. In most cases source code will be provided in the repository: surfsnippets. I don’t like to read long posts so I will write just enough to describe the experiment. I follow the principle: “The harder to write, the harder to read”.

I would like to thank Jeanna Matthews who helped me to get to Computer Science and wrote a wonderful book on Computer Networking.