dexity.github.io

Eucaby Architecture

Tue, 15 Sep 2015 00:00:00 -0700

Eucaby consists of three separate applications. The core is Eucaby API which serves messages between users and integrates external web services. Mobile application is the main client which sends and receives user location messages. Eucaby website provides general information and allows to view messages sent to email address.

Before we start feel free to download Eucaby mobile application from Google Play or Apple AppStore and check out the website www.eucaby.com.

Eucaby API

Overview

Eucaby API runs on Google AppEngine which provides scalable infrastructure and essential architecture components for real-time communication. A real-time messaging consists of several steps:

User sends location message from mobile device (or browser).
Message is stored in Cloud Datastore and notification is sent to Task Queue.
Notification in Task Queue is processed and sent to push notification service (GCM or APNs).
Notification is delivered to the recipient mobile device.

Components

Eucaby API uses three types of storage: Cloud SQL, Cloud Datastore and Memcache. Cloud SQL is mostly used to store high integrity data such as user and device data. Cloud Datastore, a NoSQL database, stores user messages and locations which don’t require high data integrity or complex queries but require high scalability and performance. Memcache, a key-value storage, is mostly used to cache data from proxy requests to external services such as Facebook Graph API or Google Maps Distance Matrix API.

Operations which can be handled outside of the request are precessed in the Task Queue. This includes push notifications, email notifications and cron jobs. Here is the example of notification task to GCM:

import flask
import logging
from flask import views
from flask import current_app
from gcm import gcm
from sqlalchemy import exc

from eucaby_api import args as api_args
from eucaby_api import models
from eucaby_api.utils import utils as api_utils


class GCMNotificationsTask(PushNotificationsTask):

    """Push notifications for GCM."""

    def post(self):  # pylint: disable=no-self-use
        res = self.handle_input(api_args.ANDROID)
        if isinstance(res, flask.Response):
            return res

        regs = {}
        for dev in flask.request.devices:
            regs[dev.device_key] = dev

        data = api_utils.gcm_payload_data(
            flask.request.sender_name, flask.request.message_type,
            flask.request.message_id)
        gcm_app = gcm.GCM(current_app.config['GCM_API_KEY'])
        try:
            resp = gcm_app.json_request(
                registration_ids=regs.keys(), data=data, retries=7)
        except gcm.GCMException as e:
            logging.error(
                'Failed to push notification. %s: %s',
                e.__class__.__name__, e.message)
            return e.message, 500

        msg = 'GCM result: {}'.format(str(resp))
        if 'errors' in resp:
            for error, reg_ids in resp['errors'].items():
                if error in ['NotRegistered', 'InvalidRegistration']:
                    # Deactivate multiple devices
                    for reg_id in reg_ids:
                        regs[reg_id].deactivate()

        if 'canonical' in resp:
            for reg_id, canonical_id in resp['canonical'].items():
                # Replace registration_id with canonical_id
                device = regs[reg_id]
                device.device_key = canonical_id
                models.db.session.add(device)
                try:
                    # Note: From time to time GCM garbage collects registration
                    #       ids by issuing a new canonical id which replace
                    #       other registration ids for the same device. You
                    #       need to deactivate the devices
                    models.db.session.commit()
                except exc.IntegrityError:
                    models.db.session.rollback()
                    device.deactivate()
        return msg

Authentication

Authentication Flow Diagram

Facebook implements non-standard OAuth with two types of access tokens: short-lived (sl_access_token) and long-lived (ll_access_token) described in Facebook Access Tokens. Eucaby API authentication uses Facebook authentication as follows:

Request short-lived access token from Facebook using username and password.
Exchange short-lived access token for long-lived access token and store it for further Facebook Graph API requests.
Create Eucaby access token and return it back to mobile device for further Eucaby API requests.

Authentication Components

Every access token has limited lifetime. This equally relates to Facebook short-lived and long-lived and Eucaby access tokens. After access token is expired it has to be either refreshed or created again. Eucaby API takes care of it by automatically refreshing access tokens.

Endpoints

Originally I planned to use Google Endpoints because it is really easy to implement API with. But it has one disadvantage that is hard to get around: all endpoints will start with /_ah/api which is no problem for programming interface but makes the REST API not very esthetic. One way to solve the issue is to use redirect for every request but I decided to take a different approach and use Flask Restful instead. It has all essential features to build API but lacking a bit of validation.

Mobile Application

Mobile application is implemented with the popular Ionic framework which is one layer on the top of PhoneGap. It is really easy to use if you get along with AngularJS and allows to generate native code for several mobile platforms out of the box. It supports plugin architecture so you can use pretty much any mobile native functionality with a simple javascript interface. I used a few plugins including PushPlugin for receiving push notifications from GCM or APNs and Geolocation for accurate geographic location. It is perfect for rapid prototyping and conveying ideas.

I noticed a few disadvantages building mobile application with Ionic compared to native application:

Larger memory footprint. PhoneGap runs embedded browser.
Slower.
Quirky. Some interactions look unusual.

AngularJS module which communicates with Eucaby API application is EucabyApi. It uses OpenFB module to authenticate with Facebook OAuth. Mobile application doesn’t use Facebook Graph API directly though. Here is the snippet of the code for EucabyApi module:

angular.module('eucaby.api', [
    'openfb',
    'eucaby.utils',
    'eucaby.config'
])

.factory('EucabyApi', [
    '$http',
    '$q',
    'OpenFB',
    'utils',
    'storageManager',
    'config',
function ($http, $q, OpenFB, utils, storageManager, config) {

    return {
        init: function(){
            OpenFB.init(config.FB_APP_ID,
                        'http://localhost:8100/oauthcallback.html');
        },
        login: function(force){
            var deferred = $q.defer();
            var accessToken = storageManager.getAccessToken();
            if (accessToken && !force) {
                deferred.resolve(accessToken);
            } else {
                /*
                * Get short-lived Facebook access token
                * Get Facebook user profile with '/me' request
                * Get Eucaby access token
                */
                // ...

                OpenFB.login('email,user_friends,public_profile')
                    .then(function(){
                        // ...
                    });
            }
            return deferred.promise;
        },
        api: function(obj){
            /*
            * Ensure that Eucaby access token exists
            * Refresh Eucaby access token if it is expired
            * Make API request
            */
        
            var self = this;
            var method = obj.method || 'GET';
            var params = obj.params || {};
            var data = obj.data && utils.toPostData(obj.data) || '';
            var deferred = $q.defer();
            
            // ...
            
            var apiRequest = function(method, path, token, params){
                return $http({
                    method: method, url: config.EUCABY_API_ENDPOINT + path,
                    params: params, data: data,
                    headers: {'Authorization': 'Bearer ' + token}});
            };
            
            var makeApiRequest = function(data){
                // ...
                apiRequest(method, obj.path, token, params)
                    .success(function(data){
                        deferred.resolve(data);
                    });
            };

            self.login().then(makeApiRequest);
            return deferred.promise;
        },
        logout: function(){
            // ...
            OpenFB.logout();
        }
    };
}]);

Eucaby Website

Eucaby website application not only displays general information but also allows users to view or send location message back to the sender. It is implemented in Django and uses some shared resources with Eucaby API.

I am here

Eucaby: Simple Geo Messenger

Tue, 01 Sep 2015 00:00:00 -0700

Messenger is no longer about just sending text. In recent years it turned into a major tool of communication for photo, video and audio content in real time. Integration of various services and business applications became indispensable part of messenger platform. Eucaby takes the messenger evolution to the next level by adding geographic location.

Eucaby is a simple Geo Messenger that helps you to share and request accurate and authenticated location messages with your friends. It is a cross-platform mobile application available for Android and iOS. Feel free to download Eucaby mobile application from Google Play or Apple AppStore and check out the website www.eucaby.com. Here we discuss the main Eucaby features.

Authentication

Eucaby supports Facebook authentication. To login, you click the Login with Facebook button and go through the standard Facebook authentication. Eucaby requests only basic information such as user profile and list of friends who use the application.

Sending Location and Request

To send location click I am here button and the dialog will open with the optional message field, email field and list of friends. Only your Facebook friends who use the Eucaby application will be listed there. Map in the dialog displays your current location which is relatively accurate depending on your phone settings. It is advised to enable Wi-Fi for better accuracy. Setting manual location will be supported in the future.

You can send location either to any email address or your friend. Sending location via email is one of the most flexible ways to share your location. Email recipient will receive the link with your current location which is valid for 1 day. She doesn’t need to have Eucaby application installed to view the location. You can even send location to your email address to remember where you have been :). Sending location via friend contact will promptly notify your friend about your current location through the application and will not share it on Facebook.

Recent friends or emails will be displayed in recent contacts (up to 3) to make it convenient for you to find the recent contact. Email input field supports autocomplete, so when you start typing email it will try to autocomplete the email addresses that you used before.

To send request use Where are you? button and in the open dialog you can ask your friend about her current location. She can respond back with the current location either from the application (preferred) or following the link which opens the location form in the browser. Request link will be valid for 1 day. Although recipient can view your location or even send her location back to your request, the requests and initiating locations can only be sent from the Eucaby mobile application.

… and don’t forget to pass a message along :).

Incoming and Outgoing Messages

Incoming and Outgoing tabs keep track of your location or request messages that you received from (Incoming) or sent to (Outgoing) other users. The most frequently used tab will probably be Incoming tab. When you send location or request message, it will be displayed with outline location or bolt icon in Outgoing tab respectively. After user responds with location to your request it will be displayed in Incoming tab with filled location icon. You still can see your original request in Outgoing tab and it will be displayed with filled bolt. Filled icon is a sign that your location or request are complete. So when you see your request complete you don’t feel alone :). After you received a request from other user you can send back your current location.

Well, a few more locations for the same request are also possible :).

Settings

By default, when someone sends you request or location you will receive an email notification. If you are a popular person and getting too many of them you might want to consider turn off the email notifications from Settings menu.

I am here

Non-Blocking Request Handling

Sun, 12 Feb 2012 00:00:00 -0800

Web applications normally perform various operations behind the scenes which take some time to process such as writing to remote database, logging over network file system or sending emails. Synchronous processing of the slow operations will reduce the responsiveness of the web application and make the user experience not very pleasant. Here I compare two non-blocking approaches using epoll and threading.

Code: surfsnippets/asynchandler

Suppose there is some operation in our web application which slows it down. The question is how to handle this somewhat independent operation asynchronously without blocking the response. I wrote two simple scripts which demonstrate epoll and threading and compared the benchmarks for these two approaches. In our case the slow operation is just sleeping for 1 second.

1
2
3
4
import time

def make_log(recv):
    time.sleep(1)

Blocking Request Handling

Naive approach to request handling is to perform the operations synchronously without noticing that some operations take much more time to complete than the others. Here is the code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
# sync.py

import os
import socket
import threading
import time

PORT        = 8080
HOST        = "127.0.0.1"
SOCK_FLAGS  = socket.AI_PASSIVE | socket.AI_ADDRCONFIG
counter     = 0     # global variable

def get_inet_socket(backlog=128):
    "Blocking socket"
    res     = socket.getaddrinfo(HOST, PORT, socket.AF_UNSPEC, socket.SOCK_STREAM, 0, SOCK_FLAGS)
    af, socktype, proto, canonname, sockaddr = res[0]
    sock    = socket.socket(af, socktype, proto)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    sock.bind(sockaddr)
    sock.listen(backlog)
    return sock


def make_log(recv):
    "Perform logging"
    global counter
    counter  += 1
    print "num = %s" % counter
    print recv
    time.sleep(1)


def main():
    # Create server socket
    isock   = get_inet_socket()

    while True:
        # Get data from the inet client
        conn, addr  = isock.accept()
        recv    = conn.recv(1024)

        # Blocking request handling
        make_log(recv)

        # Respond to the inet client
        conn.send("Doobie Doo")
        conn.close()

    isock.close()

if __name__ == "__main__":
    main()

Non-Blocking Request Handling with Epoll

A better approach is to use epoll system call to process the event. The epoll mechanism is used in a popular Tornado and Nginx web servers. In Tornado inet socket is registered with the epoll and list of callback handlers. When the request comes in, the epoll event is received and processed by the corresponding handler.

In my approach, I already have web server listening to this socket and some of the web servers already use the epoll mechanism. So instead of registering inet socket I use internal unix socket to talk to the epoll thread. The system looks like the following. Initially, two threads are running: main and epoll threads. The main thread listens to incoming requests and sends data to the unix socket registered with the epoll. The epoll thread (or LoggingThread) receives data from the unix socket and asynchronously handles the request (in our case it just takes a nap :) ). Here is the code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
# async_epoll.py

import os
import errno
import select
import socket
import functools
import threading
import time

_EPOLLIN    = 0x001
_EPOLLERR   = 0x008
_EPOLLHUP   = 0x010

PORT        = 8080
HOST        = "127.0.0.1"
TIMEOUT     = 3600
SOCK_FLAGS  = socket.AI_PASSIVE | socket.AI_ADDRCONFIG
EPOLL_FLAGS = _EPOLLIN | _EPOLLERR | _EPOLLHUP
SOCK_NAME   = "/tmp/logger.sock"
counter     = 0     # global variable

class LoggerThread(threading.Thread):

    def __init__(self):
        threading.Thread.__init__(self)


    def run(self):
        sock    = get_server_socket()
        ep      = select.epoll()
        ep.register(sock.fileno(), EPOLL_FLAGS)         # register socket
        handler = functools.partial(conn_ready, sock)   # add handler for the socket

        events      = {}
        while True:
            event_pairs = ep.poll(TIMEOUT)
            events.update(event_pairs)
            while events:
                fd, ev = events.popitem()
                try:
                    handler(fd, ev)
                except (OSError, IOError), e:
                    if e.args[0] == errno.EPIPE:
                        pass


def handle_connection(conn, address):
    "Handles connection"
    make_log(conn.recv(1024))


def conn_ready(sock, fd, ev):
    while True:
        try:
            conn, address = sock.accept()
        except socket.error, e:
            if e.args[0] not in (errno.EWOULDBLOCK, errno.EAGAIN):
                raise
            return
        conn.setblocking(0)
        handle_connection(conn, address)


# Unix socket
def get_server_socket(backlog=128):
    "Server for unix socket which listens for connections"
    sock    = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    sock.setblocking(0)
    try:
        os.unlink(SOCK_NAME)    # Clean up socket
    except:
        pass
    sock.bind(SOCK_NAME)
    sock.listen(backlog)
    return sock


def get_client_socket():
    "Client for unix socket"
    sock    = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
    return sock


# Inet socket
def get_inet_socket(backlog=128):
    "Blocking inet socket"
    res     = socket.getaddrinfo(HOST, PORT, socket.AF_UNSPEC, socket.SOCK_STREAM, 0, SOCK_FLAGS)
    af, socktype, proto, canonname, sockaddr = res[0]
    sock    = socket.socket(af, socktype, proto)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    sock.bind(sockaddr)
    sock.listen(backlog)
    return sock


def make_log(recv):
    "Perform logging"
    global counter
    counter  += 1
    print "counter = %s" % counter
    print recv
    time.sleep(1)


def main():
    # Create Logger thread
    t   = LoggerThread()
    t.setDaemon(True)
    t.start()

    # Create server socket
    isock   = get_inet_socket()

    while True:
        # Get data from the inet client
        conn, addr  = isock.accept()
        recv    = conn.recv(1024)

        # Send received data to socket
        sock    = get_client_socket()
        sock.connect(SOCK_NAME)
        sock.send(recv)
        sock.close()

        # Respond to the inet client
        conn.send("Doobie Doo")
        conn.close()

    isock.close()

    # Wait for the thread
    t.join()


if __name__ == "__main__":
    main()

Non-Blocking Request Handling with Threads

Another approach for non-blocking request handling is to use threads. The idea is somewhat described in the post “Threading in Django”. When the request comes in, a new thread is created which handles the request. This method is used by Apache web server and requires more memory than for event driven web servers. Here is the code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
# async_thread.py

import os
import socket
import threading
import time

PORT        = 8080
HOST        = "127.0.0.1"
SOCK_FLAGS  = socket.AI_PASSIVE | socket.AI_ADDRCONFIG
counter     = 0     # global variable

class LoggerThread(threading.Thread):

    def __init__(self):
        threading.Thread.__init__(self)
        self._recv  = None


    def set_recv(self, recv):
        self._recv  = recv

    def run(self):
        make_log(self._recv)


# Inet socket
def get_inet_socket(backlog=128):
    "Blocking socket"
    res     = socket.getaddrinfo(HOST, PORT, socket.AF_UNSPEC, socket.SOCK_STREAM, 0, SOCK_FLAGS)
    af, socktype, proto, canonname, sockaddr = res[0]
    sock    = socket.socket(af, socktype, proto)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    sock.bind(sockaddr)
    sock.listen(backlog)
    return sock


def make_log(recv):
    "Perform logging"
    global counter
    counter  += 1
    print "counter = %s" % counter
    print recv
    time.sleep(1)


def main():
    # Create server socket
    isock   = get_inet_socket()

    while True:
        # Get data from the inet client
        conn, addr  = isock.accept()
        recv    = conn.recv(1024)

        # Create Logger thread
        t   = LoggerThread()
        t.set_recv(recv)
        t.setDaemon(True)
        t.start()

        # Respond to the inet client
        conn.send("Doobie Doo")
        conn.close()

    isock.close()


if __name__ == '__main__':
    main()

Benchmarks

Now we can do some benchmarks for synchronous, asynchronous with epoll and asynchronous with threads approaches. For benchmarking I used popular Apache Benchmark ab tool. I start one of the scripts in one terminal, like:

[terminal 1]$ python asynch_epoll.py

[terminal 2]$ ab -n 100 -c 10 http://localhost:8080/

The benchmark result for request handling with epoll will look like the following. Here we get 12345 req/sec.

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient).....done


Server Software:
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        0 bytes

Concurrency Level:      10
Time taken for tests:   0.008 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      1000 bytes
HTML transferred:       0 bytes
Requests per second:    12345.68 [#/sec] (mean)
Time per request:       0.810 [ms] (mean)
Time per request:       0.081 [ms] (mean, across all concurrent requests)
Transfer rate:          120.56 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:     0    0   0.2      0       1
Waiting:        0    0   0.1      0       1
Total:          0    1   0.2      1       1

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      1
  75%      1
  80%      1
  90%      1
  95%      1
  98%      1
  99%      1
 100%      1 (longest request)

Performing similar benchmarks for other scripts we can create the table:

	No Handling	Synchronous	Asynchronous Epoll	Asynchronous Threads
Script	`nohandling.py`	`sync.py`	`async_epoll.py`	`async_thread.py`
Req/sec	10000	1	12000	3500

Looking at the table we see that synchronous request handling gives the worst req/sec. The best performance is achieved by asynchronous request handling with epoll. Though in this method the requests are not blocked, there are a few disadvantages: a) the concurrent number of requests is limited by about 130 and b) it normally takes longer to process all the requests. The point a) can be fixed by writing request handler more carefully and reach about 1000 concurrent requests as it is implemented in Tornado. Asynchronous requests with threads gives about 4 times less responsiveness than the method with epoll the all requests are processed much faster and number of concurrent requests can be higher. Performance in asynchronous with epoll method is better than for method without request handling because the later printed the received data in the terminal.

If you need the best performance and don’t care much when the requests get handled then you better go with Asynchronous Epoll method. I you want a reasonable performance and do care when the requests get handled then Asynchronous Threads will be a better approach. In any case, blocking request handling is not a solution.

MetaHome: Interface for Contolling Home Devices

Sun, 09 Oct 2011 00:00:00 -0700

Controlling home devices from a single hand held device is a dream that will soon become real. Home devices are getting smarter and include more functionality than devices we have now. I developed a simple prototype of Android application that can manage virtual devices. The idea can be useful both for Home Automation to control devices as well as for Smart Maps to display detailed information about the objects on the map such as airports or shopping centers.

Code: https://bitbucket.org/dexity/metahome

Application Description

The application manages predefined items (Restroom, Telephone, Elevator, ATM) by adding or removing them from display. The items can represent any object on the map: controllable home device or non-controllable object on the map. The application uses 3 independent data sources: Memory, Local Database and Web Service. I implemented different data sources to explore user experience. User can select any of the data sources from the Menu. For Web Service I wrote a simple PHP script and put it on remote web server which handles incoming requests from the application. Adding item to the map is as simple as clicking on the dashed square and selecting the item.

Layout and Drawing

The layout consists of three main views: MapView, TextView and ListView.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
   android:orientation="vertical"
   android:layout_width="match_parent"
   android:layout_height="match_parent">

    <com.surfingbits.metahome.MapView
       android:id="@+id/map_view"
       android:layout_width="wrap_content"
       android:layout_height="240dip"
       android:layout_margin="10dip"/>

    <TextView android:id="@+id/status"
       android:layout_width="wrap_content"
       android:layout_height="23dip"
       android:textSize="14dip"/>

    <ListView android:id="@android:id/list"
       android:layout_width="match_parent"
       android:layout_height="wrap_content"
       android:drawSelectorOnTop="false"/>

</LinearLayout>

Both MapView and ListView are different views of the same data model described in the next section. TextView displays the status of selected data source. When the screen is rotated the application will use a different layout: layout-land/main.xml

MapView performs custom drawing of items depending if an item is located in the square or not.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
@Override
protected void onDraw(Canvas canvas)
{
    super.onDraw(canvas);

    float[] outerR = new float[] { 10, 10, 10, 10, 10, 10, 10, 10 };

    mShapeDrawable = new ShapeDrawable(new RoundRectShape(outerR, null, null));
    mShapeDrawable.getPaint().setColor(0xcccccccc);
    mShapeDrawable.setBounds(0, 0, MAPSIZE, MAPSIZE);
    mShapeDrawable.draw(canvas);

    for (int i = 0; i < ldata.size(); i++)
    {
        if (ldata.isEmpty(i))
        {
            drawPlace(canvas, i);
        }
        else {
            drawIcon(canvas, i);
        }
    }
}

Data Model and Interface

Data model of the application is very simple. It is described by a list of size 4. The index of the list represents cell index of flattened table: from left to right, top to bottom. The value of the list represents index of item which is placed in the cell. Empty cell has value -1.

Items indices:

- Restroom
- Telephone
- Elevator
- ATM

For example: ["0","-1","-1","2"]

means that item Restroom (item index 0) is in cell (0, 0) (cell index 0) and Elevator (item index 2) is in cell (1, 1) (cell index 3).

To make operations on the cells independent on the data source I implemented a general interface. All manipulation with the data storage is performed through this interface. If you want to add a new data storage you need to add also implementation of the interface.

1
2
3
4
5
6
7
8
public interface IMapData {
    abstract void set(int index, int value);
    abstract int get(int index);
    abstract int[] getAll();
    abstract void remove(int index);
    abstract boolean isEmpty(int index);
    abstract int size();
}

Cells list is then translated to ImageText class which is used by views to display image and corresponding label.

1
2
3
4
5
6
7
8
9
10
class ImageText
{
    String text;
    Bitmap icon;
    public ImageText(Bitmap icon, String text)
    {
        this.icon   = icon;
        this.text   = text;
    }
}

Data Sources

As mentioned before, I implemented 3 independent data sources each of which can be selected from the application:

Memory
Local Database
Web Service

Local Database is a simple SQLite database which resides on Android file system and has the following database schema:

$ sqlite3 cells.db

sqlite> .schema
CREATE TABLE android_metadata (locale TEXT);
CREATE TABLE cells (_id INTEGER PRIMARY KEY,cid INTEGER,cell INTEGER);
sqlite> select * from cells;
1|0|0
2|1|2
3|2|1
4|3|-1

I will show here the examples of implementation set() method for each of these data sources:

Memory

1
2
3
4
5
6
7
8
9
class MemoryMapData implements IMapData
{
    private int[] cells     = new int[MetaHome.NUM_ITEMS];

    public void set(int index, int value) {
        cells[index]    = value;
    }
    ...
}

Local Database

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
class DatabaseMapData implements IMapData
{
    private DatabaseHelper mOpenHelper;

    private static final String DBNAME = "cells.db";
    private static final int DBVERSION = 3;

    public void set(int index, int value) {

        SQLiteDatabase db = mOpenHelper.getWritableDatabase();
        ContentValues values    = new ContentValues();
        values.put(Cells.COLUMN_CELL, value);
        db.update(Cells.TABLE_NAME, values, Cells.COLUMN_ID+"=?", new String[] {Long.toString(index)});
    }
    ...
}

Web Service

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
class WebMapData implements IMapData
{

    private static final String ENDPOINT    = "http://surfingbits.com/metahome/index.php";
    private static final String KEY         = "access_key";

    public void set(int index, int value) {
        HashMap<String, String> params  = new HashMap<String, String>();
        params.put("action", "set");
        params.put("cell", Long.toString(index));
        params.put("value", Long.toString(value));
        JSONObject js   = getContentObject(params);

        boolean status;
        try {
            status  = js.getBoolean("status");
            Log.i("Web", "Set value cell["+index+"] = "+value+": "+status);
        } catch (Exception e) {
            Log.e("WebError", e.toString());
        }
    }
    ...
}

Web Service Interface and Implementation

For Web Service I implemented a simple PHP script which handles incoming requests from the application. In the backend the script also uses SQLite database with the same database schema as Android is using for the Local Database source.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
<?php

// Copyright 2011 Alex Dementsov

$key        = "access_key";
$actions    = array("get", "getall", "remove", "set", "size", "empty");

class Handler
{
    private $db;

    public function __construct(){
        $this->open_db();
    }

    function run()
    {
        global $key, $actions;

        if (!(isset($_GET["key"]) && $_GET["key"] == $key) ||
            !(isset($_GET["action"]) && in_array($_GET["action"], $actions)))
        {
            echo "Wrong key or action";
            return;
        }

        switch($_GET["action"])
        {
            case "get":
                return $this->get();
            case "getall":
                return $this->get_all();
            case "remove":
                return $this->remove();
            case "set":
                return $this->set();
            case "size":
                return $this->size();
            case "empty":
                return $this->isempty();
        }
    }
    function get()
    {
        if (!isset($_GET["cell"]))
            return $this->status_fail();

        $row    = $this->getValue();
        if (!$row)
            return $this->status_fail();

        return json_encode(array("cell" => $row["cid"], "value" => $row["cell"]));
    }


    function get_all()
    {
        $array  = array(-1, -1, -1, -1);
        $rows   = $this->getValues();
        if (!$rows)
            return $this->status_fail();

        foreach ($rows as $i => $value)
        {
            $idx = $value["cid"];
            if ($idx >= 0 && $idx < 4)
                $array[$idx] = $value["cell"];
        }
        return json_encode(array("cells" => $array));
    }

    function remove()
    {
        if (!isset($_GET["cell"]))
            return $this->status_fail();

        return $this->setValue(-1);
    }

    function set()
    {
        if (!isset($_GET["cell"]) or !isset($_GET["value"]))
            return $this->status_fail();

        return $this->setValue($_GET["value"]);
    }

    private function update($arr)
    {
        // Executes query and returns status
        $query  = "update cells set cell=? where cid=?";
        $q      = $this->db->prepare($query);
        $q->execute($arr) or die(print_r($this->db->errorInfo(), true));
        return $q;
    }

    private function getValue()
    {
        // Returns value of a cell
        $result = $this->db->query("select * from cells where cid=".$_GET["cell"]) or die($this->status_fail());
        return $result->fetch();
    }

    private function getValues()
    {
        // Returns value of a cell
        $result = $this->db->query("select * from cells limit 4") or die($this->status_fail());
        return $result->fetchAll();
    }

    private function setValue($value)
    {
        $arr    = array($value, $_GET["cell"]);
        $q      = $this->update($arr);
        if (!$q)
            return $this->status_fail();

        return $this->status_ok();
    }

    function size()
    {
        return json_encode(array("size" => 4));
    }

    function isempty()
    {
        $row    = $this->getValue();
        if (!$row)
            return $this->status_fail();

        if ($row["cell"] == -1)
            return json_encode(array("empty" => True));

        return json_encode(array("empty" => False));
    }

    private function status_fail()
    {
        return $this->status("fail");
    }

    private function status_ok()
    {
        return $this->status("ok");
    }

    private function status($status)
    {
        return json_encode(array("status" => $status));
    }

    private function open_db()
    {
        try {
            $this->db = new PDO('sqlite:db/cells.db');
        }
        catch(Exception $e) {
            die($this->status_fail());
        }
    }
}

$handler = new Handler();
echo $handler->run();
?>

Here is the interface for the Web Service:

Get All Cells

Example Request:
http://HOSTNAME/metaservice/index.php?action=getall&key=access_key

Example Response:
{
    "cells": ["2", "3", "-1", "-1"]
}

Get Cell

Example Request:
http://HOSTNAME/metaservice/index.php?action=get&cell=3&key=access_key

Example Response:
{
    "cell": 3,
    "value": 2
}

Remove Cell

Example Request:
http://HOSTNAME/metaservice/index.php?action=remove&cell=3&key=access_key

Example Response:
{
    "status": "ok"
}

Set Cell

Example Request:
http://HOSTNAME/metaservice/index.php?action=set&cell=3&value=2&key=access_key

Example Response:
{
    "status": "ok"
}

Get Cells Size

Example Request:
http://HOSTNAME/metaservice/index.php?action=size&key=access_key

Example Response:
{
    "size": 4
}

Is Empty

Example Request:
http://HOSTNAME/metaservice/index.php?action=empty&cell=3&key=access_key

Example Response:
{
    "empty": true
}

If something goes wrong the response returns error:

{
    "status": "fail"
}

Caching Mechanism and Other Trick

Using Web Service data source without caching makes user experience not very pleasant. I implemented some caching mechanism to make experience better. Remote calls are made to Web Service only when it is absolutely necessary, for example to change the state of the cells. Other requests are made locally from the stored data attribute ldata.

1
2
3
4
5
public class MapView extends View {

    private IMapData ldata;         // Local storage for the data (cache)
    ...
}

This attribute is used for all three data sources even when it somewhat duplicates storage (as in Memory data source). This simple caching works surprisingly well for Web Service data source.

Other issue I had with the application is the screen rotation. When you rotate the mobile phone all the local objects are gone because Android restarts the running Activity unless you take some measures to keep the state. Here is the reference to Android documentation which discusses this issue: http://developer.android.com/guide/topics/resources/runtime-changes.html

To do the trick I implemented onSaveInstanceState() and onRestoreInstanceState(Parcelable state).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
public class MapView extends View {

    @Override
    protected Parcelable onSaveInstanceState() {
        Bundle b = new Bundle();

        Parcelable s = super.onSaveInstanceState();
        b.putParcelable("map_super_state", s);
        b.putIntArray("map_cells", dumpCells(ldata));
        b.putIntArray("memory_cells", dumpCells(memory));
        return b;
    }

    @Override
    protected void onRestoreInstanceState(Parcelable state)
    {
        Bundle b = (Bundle) state;
        Parcelable superState = b.getParcelable("map_super_state");
        loadCells(ldata, b, "map_cells");
        loadCells(memory, b, "memory_cells");

        super.onRestoreInstanceState(superState);
    }
    ...
}

Real Example

To show the possible application of the idea I present here the map of the Terminal 4 at Los Angeles International Airport (LAX). It looks a bit more complicated than what I showed you before :).

Analysis of TuneIn Radio Stations

Fri, 08 Jul 2011 00:00:00 -0700

Recently I was interested in TuneIn (http://tunein.com) radio service which provides access to radio stations around the globe. In this post I will uncover statistics behind TuneIn radio stations and show how to use the script to do some cool thinks with the TuneIn API.

Code: surfsnippets/stationdigger

Output: station_lang.json (6.9 Mb), station_genre.json (6.4 Mb)

I wrote a script stationdigger.py which uses TuneIn API http://inside.radiotime.com/developers/api/opml by traversing tree of links and collecting information about the stations.

TuneIn radio API provides both OPML (XML-like) and JSON formats. You don’t have to have any credentials to use this service. But if you start to use the service frequently you will end up getting 403 Forbidden error. Let’s try to make the first request:

http://opml.radiotime.com/Browse.ashx

Your response will be in default OPML format:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
<opml version="1">
    <head>
        <title>TuneIn</title>
        <status>200</status>
    </head>
    <body>
        <outline type="link" text="Local Radio" URL="http://opml.radiotime.com/Browse.ashx?c=local" key="local"/>
        <outline type="link" text="Music" URL="http://opml.radiotime.com/Browse.ashx?c=music" key="music"/>
        <outline type="link" text="Talk" URL="http://opml.radiotime.com/Browse.ashx?c=talk" key="talk"/>
        <outline type="link" text="Sports" URL="http://opml.radiotime.com/Browse.ashx?c=sports" key="sports"/>
        <outline type="link" text="By Location" URL="http://opml.radiotime.com/Browse.ashx?id=r0" key="location"/>
        <outline type="link" text="By Language" URL="http://opml.radiotime.com/Browse.ashx?c=lang" key="language"/>
        <outline type="link" text="Podcasts" URL="http://opml.radiotime.com/Browse.ashx?c=podcast" key="podcast"/>
    </body>
</opml>

This is where the story begins. Traversing tree of links with type="link" you can reach links with type="audio" and collect information about the stations. For example, one branch of the link tree can look like:

http://opml.radiotime.com/Browse.ashx
+-- <outline type="link" text="Local Radio" URL="http://opml.radiotime.com/Browse.ashx?c=local" key="local"/>
    +-- <outline type="audio" text="San Diego Police Scanners: 2 (Police)" URL="http://opml.radiotime.com/Tune.ashx?id=s103434" bitrate="32" reliability="86" guide_id="s103434" subtext="English" genre_id="g2741" formats="mp3" item="station" image="http://radiotime-logos.s3.amazonaws.com/s103434q.png" now_playing_id="s103434" preset_id="s103434"/>

My script stationdigger.py just makes this task automatic and more error prone.

Results Format

I collected statistics for radio stations arranged by language and genre. The results are stored in files station_lang.json (6.9 Mb) and station_genre.json (6.4 Mb) in JSON format for stations by language and genre correspondingly.

Stations by language has the format:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
{<language id>:
    {"name": <language name>,
    "stations":
        {<station id>:
            {"name": <station name: {text}>,
            "description": <station description: {subtext}>,
            "url": <station url: {URL}>,
            "image": <station image: {image}> ,
            "genre": <genre id: {genre_id}>
            },
        // other stations
        }
    }
// other languages
}

For example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
{"l191":
    {"name": "Afrikaans",
     "stations":
        {"s101967":
            {"name": "Radio Namakwaland (Vredendal, South Africa)",
             "description": "Afrikaans",
             "url": "http://opml.radiotime.com/Tune.ashx?id=s101967&filter=l191",
             "image": "http://radiotime-logos.s3.amazonaws.com/s101967q.png",
             "genre": "g133",
            },
        // ...
        }
    }
// ...
}

Parameter url holds the link to the station stream which you save on your machine and open content of the file with some media player.

For example, open the link in the browser:

http://opml.radiotime.com/Tune.ashx?id=s124197&filter=l1

and in the Tune.ashx file you will find station stream which looks something like:

http://lush.wavestreamer.com:3680/

Open this link, for example, in Banshee Media Player (Media/Open Location) available in Ubuntu as a default media player and enjoy the station :).

Code Analysis

As I mentioned before, stationdigger.py code traverses tree of links and collects information about the stations using DFS algorithm. At first I thought that it will be pretty straightforward to implement the algorithm, but later I realized that I have to handle several issues specific to TuneIn service to properly do it.

TuneIn radio service significantly limits number of calls without partnerId passed (http://inside.radiotime.com/developers/api/opml/security-model#limits). It also has limits for calls even with partnerId but the limit is larger than without partnerId. Once the limit is reached you will start getting 403 Forbidden error for a day. Because of the large number of calls needed to collect the data it is good to have partnerId :) or you can wait for a long time. To collect stations using partnerId it took me about 4-5 days. It is also recommended to pass the serial parameter which can be pretty much anything.

The core of the algorithm is method traverse():

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
def traverse(self, url):
    "Traverses tree of links and populates stations using DFS algorithm"
    if not url or (self.numTotal and self.numTotal<self.num):
        return

    items = self.make_call(url=url)
    (audios, links) = self._separate(items)
    for item in audios:    # check audio links first: playable content
        if item.has_key("guide_id") and \
           not self._stations.has_key(item.get("guide_id")):  # ignore duplicates
            self._stations[item.get("guide_id")]    = self.to_station(item) # set station
            self.num += 1
            if self.numTotal and self.numTotal<self.num:
                return

    for item in links:      # check links
        self.traverse(item.get("URL"))

In this method it makes call to specified url and returns list of items in the tag. Then it separates links and audio links. Links can later be traversed down the link tree. Audio links are what we are after: they are used to collect information about the station.

The code supports the following features.

Can dig stations both for one and all languages

Because digging stations is a relatively long process, when something happens during the data collection you need a way to restart from some intermediate step. For this purpose the program stores stations in junks: one language in a separate file in a format lang<number>.json

These files are stored in stations directory and then later merged with merge_langs.py script into a single file: station_lang.json. Besides collecting stations for all languages at once you can collect stations just for one language by passing a language tuple.

1
2
3
4
5
6
7
8
def dig_stations_lang(self, filename=None, lang_tuple=None):
    "Diggs for stations with language filters"
    if lang_tuple:
        filters = [lang_tuple]  # one language
    else:
        langjs = self.get_lang() # all languages
        filters = self.sorted_tuple(langjs)
    self._dig_stations(filters, {"c": "lang"}, filename)

Can dig specified number of stations

You don’t need to collect all stations. Setting limit on total number of stations (self.numTotal) will collect at most this number of stations for the filter passed. See implementation of method traverse(self, url). For example, if filter has two languages then stations will get populated starting from the first language.

Digs only unique stations

To make sure that stations are not duplicated for the same language the program checks if the station has been added before:

1
2
3
4
5
6
def traverse(self, url):
        ...
        if item.has_key("guide_id") and \
           not self._stations.has_key(item.get("guide_id")):  # ignore duplicates
            self._stations[item.get("guide_id")]    = self.to_station(item) # set station
        ...

Handles `403 Forbidden` error

This error was a frequent companion during collecting data. This just means that limit of using the TuneIn service has been reached. It not handled properly you will end up either with non-complete stations or no stations at all. I did some experiments with this error and figured out that your program can be rejected either for one day or for a few minutes (not sure how this happens). So I developed some mechanism that handles this error which can be reduced to just one word: persistence! When program receives 403 Forbidden error it waits for some time and then tries to perform the same call again until it receives result other than this error.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
def make_call(self, url):
    "Makes call and returns response in json format"
    if not url:
        return None
    nurl    = self._set_format(url)
    if nurl in self._urls:  # Avoid circular reference
        return None
    result  = -1    # to enter the loop

    while (result == -1):
        try:
            req         = urllib2.Request(nurl, headers=self._headers())
            response    = urllib2.urlopen(req)
            page        = self._decompress(response.read()) # decompress
            pdict       = json.loads(page)
            result      = pdict.get("body")    # interested in "body" section only: should be list
            self._urls.add(nurl)    # URL is processed
        except urllib2.HTTPError, e:
            if e.code == 403:
                result = -1
            else:
                result = None
        except:
            result = None

        if result == -1:
            print "Waiting %s --> %s" % (self.num, nurl)
            time.sleep(WAIT_TIME)
    return result

Avoids circular reference

From experience I found out that some links might have circular reference: referring to itself in one or several hops. Once the program enters the circular reference it can get stuck forever! Simple check can save you out of trouble:

1
2
3
4
5
def make_call(self, url):
    ...
    if nurl in self._urls:  # Avoid circular reference
        return None
    ...

Response compression

To optimize the stream of responses I added header to request encoded content supported by most of the popular web servers and decompress it on the program’s side:

Accept-Encoding:  gzip

Here is the method which performs decompression:

1
2
3
4
5
def _decompress(self, s):
    "Decompresses string"
    sio     = StringIO(s)
    file    = GzipFile(fileobj=sio)
    return file.read()

Handles embedded links

Another issue I had related to TuneIn responses is embedded tags which looked like in xml format:

<outline text="Stations" key="stations">
    <outline type="audio" text="Radio Pretoria (Kleinfontein)" URL="http://opml.radiotime.com/Tune.ashx?id=s10616&partnerId=my_partner_id&serial=123&filter=l191" bitrate="32" reliability="96" guide_id="s10616" subtext="Tienie se musiek" genre_id="g255" formats="wma" show_id="p149910" item="station" image="http://radiotime-logos.s3.amazonaws.com/s10616q.png" current_track="Tienie se musiek" now_playing_id="s10616" preset_id="s10616"/>
</outline>

To handle this issue I traverse also these subtrees:

1
2
3
4
5
6
7
8
9
10
11
12
def _traverse_outline(self, items, audios, links):
    "Traverses children outline elements and populate audios and links lists"
    if not items:   # No items
        return

    for item in items:
        if item.has_key("children") and item.get("children"):
            self._traverse_outline(item.get("children"), audios, links)
        elif item.get("type") == "audio":
            audios.append(item)
        elif item.get("type") == "link":
            links.append(item)

Mimics headers of Chrome browser

For some reason passing browser headers with the call doesn’t reject calls as fast as without the headers. So I decided to explore Chrome browser headers and pretend that the calls are made with the browser. Here is the example of the headers presented as a Python dictionary:

1
2
3
4
5
6
7
8
9
headers = {
    "Connection":       "keep-alive",
    "Cache-Control":    "max-age=0",
    "User-Agent":       "Mozilla/5.0 (X11; Linux i686) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.71 Safari/534.24",
    "Accept":           "application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5",
    "Accept-Encoding":  "gzip",
    "Accept-Language":  "en-US,en;q=0.8",
    "Accept-Charset":   "ISO-8859-1,utf-8;q=0.7,*;q=0.3"
}

Using Script

Before you start running program it is recommended to set config.txt file first:

[general]
partnerId =
serial =

To collect stations for all languages you need to iterate over sorted languages:

1
2
3
4
5
sd      = StationDigger()
langjs  = json.load(open("generated/lang.json"))
lang    = sd.sorted_tuple(langjs)
for n in range(len(lang)):
    sd.dig_stations_lang("stations/lang%s.json" % n, lang_tuple=lang[n])

so the session for calls without partnerId passed looks like:

Finished: Aboriginal, total: 2
Finished: Afrikaans, total: 12
Waiting 15 --> http://opml.radiotime.com/Browse.ashx?id=g373&filter=l210&render=json

If you want to just dig stations for n-th language then the script will do the work:

1
2
3
4
5
sd      = StationDigger()
langjs  = json.load(open("generated/lang.json"))
lang    = sd.sorted_tuple(langjs)
lang_tuple  = lang[n]
sd.dig_stations_lang("stations/lang%s.json" % n, lang_tuple=lang_tuple)

Other Useful Scripts

Besides implementing stationdigger.py I also implemented some useful scripts performing some operations on generated files.

`merge_langs.py`

This script merges files with stations by a single language into one file: station_lang.json. You need to have language files already generated from stations collections. The file structure should look like:

stations/
    lang0.json
    lang10.json
    ...

`lang2genre.py`

It turns out that the stations by language can be converted into stations by genre because most of the stations have also genre field. This way one can collect statistics of station distribution by genre. lang2genre.py script converts station_lang.json into station_genre.json.

`status_generator.py`

This script uses station_lang.json and station_genre.json to generate cumulative statistics on languages and genres. The result is later used to draw plots.

`draw_plots.py`

Script for generating pie and histogram plots for languages and genres.

Plotting Results

This is the final step in our analysis. We take data generated by status_generator.py script and use draw_plots.py to create plots for languages and genres. To draw plots I used matplotlib – an advanced plotting Python library extensively used by scientists. I used pie and histogram plot types to represent data in two different ways. I wasn’t happy with the default color schema so I wrote function which generates html-type colors from HSV colors.

Here are the results I obtained:

Results Analysis

It is no surprise that TuneIn radio service is a US based company so probably they are more biased to English language. English speaking stations comprise 42.7% of all provided stations. Next popular language, Spanish and Portuguese, have about 10% each.

Genre distribution is not that narrow as for languages. Majority of stations feature pop genre (Top 40-Pop: 15.4% and Adult Contemporary: 6.0%) which is also predictable. Detailed statistics can be found in status_lang_sort.txt and status_genre_sort.txt.

Sea World Tiles

Mon, 23 May 2011 00:00:00 -0700

After hanging out you with my friends from Oregon to the SeaWorld in San Diego, CA I decided that it is might be cool to develop a simple Android application which keeps our memory about the adventure parks.

Code: surfsnippets/shamu

So here is the app called Sea World Tiles. Feel free to download it from Android Market and play around. In this app you need to compose the picture of a sea creature from left and right sides by clicking the appropriate buttons.

The source code is available in the repository surfsnippets in directory shamu. Here is the short description of source code to show how simple it is to write Android applications.

The layout in Android applications is described in .xml file, main.xml and includes TableLayout with two TableRow. The upper TableRow displays two tiles of ImageView and the bottom TableRow keeps left and right buttons:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
   android:layout_width="fill_parent"
   android:layout_height="fill_parent"
   android:orientation="vertical"
   android:id="@+id/main_layout">
    <TableLayout android:id="@+id/tableLayout1" android:layout_width="match_parent" android:layout_height="wrap_content">
        <TableRow android:id="@+id/tableRow1" android:layout_width="wrap_content" android:layout_height="wrap_content">
            <ImageView android:layout_width="wrap_content"
                       android:layout_height="wrap_content"
                       android:src="@drawable/seaworld_l"
                       android:id="@+id/picture_l"></ImageView>
            <ImageView android:layout_width="wrap_content"
                       android:layout_height="wrap_content"
                       android:src="@drawable/seaworld_r"
                       android:id="@+id/picture_r"></ImageView>
        </TableRow>
        <TableRow android:id="@+id/tableRow2" android:layout_width="wrap_content" android:layout_height="wrap_content">
            <Button android:text="Left"
                    android:id="@+id/button_left"
                    android:layout_width="wrap_content"
                    android:layout_height="wrap_content"
                    android:layout_weight="0.5"
                    android:layout_margin="5dip"></Button>
            <Button android:text="Right"
                    android:id="@+id/button_right"
                    android:layout_width="wrap_content"
                    android:layout_height="wrap_content"
                    android:layout_weight="0.5"
                    android:layout_margin="5dip"></Button>
        </TableRow>
    </TableLayout>
</LinearLayout>

Layout specifies some important parameters such as widget id: @android:id="+id/button_left", sizes: android:layout_weight="0.5" or relation to container widget: android:layout_margin="5dip". Some of these parameters can be used in program.

The program is consists of just one Shamu.java:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
package com.surfingbits.shamu;

import java.util.HashMap;

import android.app.Activity;
import android.os.Bundle;
import android.view.View;
import android.widget.Button;
import android.widget.ImageView;

import java.util.Random;

public class Shamu extends Activity {
    /** Called when the activity is first created. */

    HashMap<Integer, String> num2pic;
    private static Random randNum;
    public static int randl, randr;

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);

        initValues();

        Button btnLeft = (Button) findViewById(R.id.button_left);  // Left button
        btnLeft.setOnClickListener(new Button.OnClickListener() {
            public void onClick(View v) {
                Shamu.this.replaceImage("l");
            }
        });
        Button btnRight = (Button) findViewById(R.id.button_right); // Right button
        btnRight.setOnClickListener(new Button.OnClickListener() {
            public void onClick(View v) {
                Shamu.this.replaceImage("r");
            }
        });
    }

    public void initValues()
    {
        randNum = new Random();
        randl   = 0;
        randr   = 0;
        num2pic = new HashMap<Integer, String>();
        num2pic.put(new Integer(0), "shamu");
        num2pic.put(new Integer(1), "dolphin");
        num2pic.put(new Integer(2), "seastar");
        num2pic.put(new Integer(3), "rayfish");
    }

    public int getRandNum(String side)
    {
        // Two static variables randl and randr are set by corresponding button
        try {
            int temp    = Shamu.class.getField("rand"+side).getInt(null);   // randr or randl
            int tempB   = temp;
            while (tempB == temp)
            {   // try until you get different int
                temp    = randNum.nextInt(num2pic.size());
            }
            Shamu.class.getField("rand"+side).setInt(Shamu.class, temp);
            return temp;
        } catch (NoSuchFieldException e) {
            return 0;
        } catch (SecurityException e) {
            return 0;
        } catch (IllegalAccessException e) {
            return 0;
        }
    }

    public int layoutId(String base, Class cls, String side)
    {
        // Trying to use reflection getField (similar to Python getattr) to get field value
        int id;
        // Check if side is "l" or "r"
        try {
            id  = cls.getField(base+"_"+side).getInt(null); // Example: R.id.picture_l
        } catch (NoSuchFieldException e) {
            id = 0;
        } catch (SecurityException e) {
            id = 0;
        } catch (IllegalAccessException e) {
            id = 0;
        }
        return id;
    }

    public void replaceImage(String side)
    {
        ImageView pic   = (ImageView) findViewById(layoutId("picture", R.id.class, side));
        pic.setImageResource(layoutId(num2pic.get(getRandNum(side)), R.drawable.class, side));
    }
}

When Android application is started the onCreate() method is called. First, it loads the main layout:

setContentView(R.layout.main);

initializes attributes and and then creates button widget specified by id R.id.button_left and defines the OnClickListener():

1
2
3
4
5
6
Button btnLeft = (Button) findViewById(R.id.button_left);  // Left button
btnLeft.setOnClickListener(new Button.OnClickListener() {
    public void onClick(View v) {
        Shamu.this.replaceImage("l");
    }
});

When the user clicks the button replaceImage() method will be called:

1
2
3
4
5
public void replaceImage(String side)
{
   ImageView pic    = (ImageView) findViewById(layoutId("picture", R.id.class, side));
   pic.setImageResource(layoutId(num2pic.get(getRandNum(side)), R.drawable.class, side));
}

This method takes the ImageView by id and replaces it with another image by randomly generating number and looking up the appropriate id in the HashMap for the next picture.

To make the source code more compact I used Java Reflection: Reflection (computer programming) that can refer to class attributes or methods by name string. This approach is similar to Python functions getattr() or setattr().

Hello world

Sun, 20 Feb 2011 00:00:00 -0800

In this blog I will explore cool technologies by doing some experiments on computer systems, small and large, and writing code. In most cases source code will be provided in the repository: surfsnippets. I don’t like to read long posts so I will write just enough to describe the experiment. I follow the principle: “The harder to write, the harder to read”.

dexity.github.io

Eucaby Architecture

Eucaby API

Overview

Components

Authentication

Authentication Flow Diagram

Authentication Components

Endpoints

Mobile Application

Eucaby Website

Eucaby: Simple Geo Messenger

Authentication

Sending Location and Request

Incoming and Outgoing Messages

Settings

Non-Blocking Request Handling

Blocking Request Handling

Non-Blocking Request Handling with Epoll

Non-Blocking Request Handling with Threads

Benchmarks

MetaHome: Interface for Contolling Home Devices

Application Description

Layout and Drawing

Data Model and Interface

Data Sources

Memory

Local Database

Web Service

Web Service Interface and Implementation

Get All Cells

Get Cell

Remove Cell

Set Cell

Get Cells Size

Is Empty

Caching Mechanism and Other Trick

Real Example

Analysis of TuneIn Radio Stations

Results Format

Code Analysis

Can dig stations both for one and all languages

Can dig specified number of stations

Digs only unique stations

Handles 403 Forbidden error

Avoids circular reference

Response compression

Handles embedded links

Mimics headers of Chrome browser

Using Script

Other Useful Scripts

merge_langs.py

lang2genre.py

status_generator.py

draw_plots.py

Plotting Results

Results Analysis

Sea World Tiles

Hello world

Handles `403 Forbidden` error

`merge_langs.py`

`lang2genre.py`

`status_generator.py`

`draw_plots.py`