A new life for vocabup.com: MARKETING

I reached a new stage in developing the vocabulary enhancement app. I come to a point to stop development for a while and do market research and marketing. As in case of vocabup.com I am marketing and development guy in one person and I realize now, that programming does not make the whole show. The previous work was good and I reached my goal to build a reference app showing state of the art software development implementing the progressive enhancement and mobile first webdesign principle. The result is a very accessible app that can be enhanced up to a native equivalent offline app easily. The app can even do the balance act to run without JavaScript, a characteristic of Accessibility. The effort to make the app offline is far less than to write an app for every platform. All I have to do is to write/extend the client side database layer and master and apply Application Cache. As I chose a dedicated node.js webservice as the view layer I do not have to change my view code, be it for server side rendering or client side rendering whether via Ajax or IndexedDb, since this layer can be executed on server and client. I was not under time pressure and could think in the very long term and strategically and so I was able to build the jack of all trades software, or lets say the jack of all platforms and use cases.

Look at the performance test results for the vocabulary app exercise page! There are still some minor issues. The database queries for this page are complex in order to present the user an intelligent word quiz and so it takes time for the server to send the first byte to the browser. The thesaurus pages can let users wait for a second too. There are words having up to 75 definitions each of them having synonyms, sample sentences and domains they belong to and many semantically related definitions which for their part have synonyms and sample sentences. The WortNet 3.1 database is like its name says a net, so the information you get about a word is complex and relating to other words. That is what make this thesaurus and dictionary so interesting but also stresses resources. But the majority of the vocabup.com pages load fast, like that page. I decreased the page size by compressing all the big JavaScript and CSS files and even the HTML is compressed in the case of the blog pages or React apps. Now the vocabulary app can be accessed even when the data transfer rate on a mobile device is low. That means my website is also accessible to many countries which has poor infrastructure. In Germany there are many spots where mobile services has no chance. But my app will still respond quickly! And the app fits in all displays in contrast to many old and established web apps.

I had a problem with React. This does not shed good light upon React and I was worried first. I still did not get an answer from the React guys on stackoverflow yet. Maybe I post it in the React forum more prominently. On the other side I am satisfied with a workaround in the meantime. As I have a marketing phase now anyway I will return to that problem later.

In order to start marketing I integrated a wordpress blog into vocabup.com. The strategist that I am I have different goals by writing a blog. First is search engine optimization (SEO). Search engines love text and if I want to be found by them I have to tell them how good I am at knowing and providing ways how to improve vocabulary. But it is more than search engines. I read some pages about improving vocabulary. Read, read, read, they say and write. Having a strong vocabulary is said to be the most important factor for having success. Writing really helps myself to dive into the topic and find my niche in the market. And while doing this I can praise what vocabup.com can already do for vocabulary enthusiasts and gather ideas how to improve and direct vocabup.com. Google Analytics will show me which keywords are best to concentrate on and where competition is low enough and what the visitors are interested in. Finally I can enter the market and develop the software very focused and thus efficiently. So blogging is a very cheap and efficient way to get into the market and try keyword combinations out and get feedback.

Tutorials like this helped me to find an approach how to incorporate a wordpress blog into vocabup.com seamlessly. An obstacle was the unexpected behavior openshift adds to committing and pushing a git repository to the production server. Normally you would tell git which files to leave out from committing and pushing by the .gitignore file. My aim was to just leave different wordpress installations on development and production server and maybe just check in and deploy the theme. Git supports this workflow and can ignore files that are not in the index. But Openshift deletes ignored files from the remote repository on the production server. Openshift clears the remote repository at first and push the resources freshly each time you git push to production. So I just had to write a post deployment action hook, which creates a blog directory and places a symbolic link into it which points to a persistent directory where I have my wordpress installation now. But know that!

Offline runnable Thesaurus Training Web App – JavaScript in Action

Since Web SQL is deprecated I use IndexedDB as client storage to get the offline part of my offline runnable thesaurus training web app done. I did not get memory limits yet on any browser. Just on desktop safari it takes eternity to import the data into IndexedDB. On mobile Safari it will probably take 10 times eternity. So, safari users can forget the offline part und must go the online route. Or install Chrome or Firefox. But, hey, thanks to progressive enhancement, the app will not break for you und you can still use the app with all features online. I do not suspend Safari users from thesaurus training! The CSV files which I converted the wordnet DB by a small Java program from an SQLite DB into are 16.7 MB big. The CSV files are stored as ZIP files on the server. They are downloaded and decompressed by using the JavaScript library zip.js. According to the author the library uses many interesting APIs of which Typed array (originated in WebGL) is mandatory and Web workers, File, File Writer and File Directories and System can be treated as optional. I use the whole package for now and will do the fine tuning, which means integrating older browsers, later. Only for production I had to change the code to have the file sizes of the zipped CSV files as static arguments available, since the Ajax requests for getting the Content-Length from response headers for that files failed. Since it’s static data, I have no problem doing this and I do not have to puzzle on the strange problem.
As I had read Christian Heilmanns book alone, the only thing to structure my code was the JavaScript object notation. But I found it too bad, that each and every variable and function was public. That’s not a nice interface for the small library I had written to download and uncompress the data and store it into IndexedDB and to access the data. This is not how software development works. I want to be able to read the code several years later und a clear public interface helps enormously. Here is an example model for how I am structuring my database code now:

var KasparDavid = (function () {
 // Instance stores a reference to the Singleton
 var instance;
 function init() {
   // Singleton
   // Private methods and variables
   function getName(){
       return privateNameFirst + " " + privateNameLast;
   }
   var privateNameFirst = "Kaspar";
   var privateNameLast = "David";
   return {
     // Public methods and variables
     introduceYourselfTo: function (name) {
       console.log( "Hello " + name + ", I'm " + getName());
     },
     publicFunction: "intoduceYourselfTo"
   };
 };
 return {
   // Get the Singleton instance if one exists
   // or create one if it doesn't
   getInstance: function () {
     if ( !instance ) {
       instance = init();
     }
     return instance;
   }
 };
})();
// Usage:
var singleA = KasparDavid.getInstance();
var singleB = KasparDavid.getInstance();
console.log( singleA === singleB ); // true
singleA.introduceYourselfTo("Sandra");

Like the example class KasparDavid I have a Singleton data access class which has only one instance because I do not want to have some functionality executed multiple times simultaneously. The data access class has a public interface and hides all the complex things, that have to be done to import the data and show the import state and to make complex queries that return a model for rendering.
The database script is unobtrusive since the app does not depend on the offline feature. If no JavaScript is available loading the data works with good old reloading the page. If the import is running the data requested by the user is loaded via Ajax. I like JavaScript here very much. The Ajax request fetches some JSON wich can be directly converted into a complex JavaScript object.

Offline runnable Thesaurus Training Web App – Deployment

Deployment of the offline runnable thesaurus training web app was not the easiest part so far. Yii has some configuration aids. Some portion of code may be configuration dependent. For example, the paths to the resources may vary from development to production. There are Yii variables like YII_ENV. When YII_ENV ist set to prod, then YII_ENV_PROD is defined, so you code can make a query like if (YII_ENV_PROD) { do it; }. Now I have one or two places in my code, to ask if in production or in development and set the paths accordingly.
What puzzled me most about Yii until now was the publishing of assets. An asset is a resource like a Javascript or CSS file. When they are getting auto published, what must explicitly be prohibited, they reside in folder with a hashed name. It is to avoid version conflicts. I can’t find a use case for that yet. But each time I wanted to deploy the app on openshift, the scripts where put into a folder with another hashed name. So I cannot reference them outside Yii. I am using HTML5 Application Cache, an HTML5 technique to make web sites or part of them able to run offline. For knowing which files to cache, there is a manifest file which references this files. Even if there is a server connection available, this files will be obtained from the cache, that is defined in the manifest. Thus, HTML5 Application Cache saves bandwidth. Sometimes I may want to update my Javascript files, so this requires to specify an update strategy which I left out for the future. The way to left out the resources to be published is to set the variable $sourcePath in each asset class which extends the class AssetBundle to null. Took me a while to find this out. In addition the initialization of the array js which is inherited from the AssetBundle base class depends on where the app runs, on localhost or on production server. My solution is to override the init() method of the existing asset classes and use a method getAssetSourcePath() there which I added to the AssetBundle class:

    public function getAssetSourcePath(){
        if (YII_ENV_DEV) {
            return "phpapp/web/";
        } else if (YII_ENV_PROD){
            return "";
        }
    }

My YiiAssetclass now looks like this:

class YiiAsset extends AssetBundle
{
    public $sourcePath = null;

    public function init(){
        $this->js = [];
        array_push($this->js, parent::getAssetSourcePath().'js/yii.js');
        parent::init();
    }

    public $depends = [
        'yii\web\JqueryAsset',
    ];
}

deployment on openshift is the easiest part of deployment and works with a few console commands using git, the rhc command and in my case mysql. First I have to install ruby, since the rhc command line tool requires it. The documentation on openshift can be followed quite straight forward.

Deploying the MySQL database worked with these commands:

rhc scp myapp upload /path/to/db.slp app-root/data
rhc app ssh myapp
mysql $OPENSHIFT_APP_NAME < $OPENSHIFT_DATA_DIR/db.sql

Another strange problem: To download compressed CSV files and uncompress them via Javascript, I use zip.js. In the production environment I stumbled upon a strange error. zip.js first makes a HEAD Ajax request, before actually downloading the zipped file via Ajax. But the Content-Length of the response header was not there, so Javascript reads 0 and zip.js interpretes this as a corrupted file. I don’t want to spend much time to go deep into that problem. I just return the Content-Length statically, since all data is static. Sure, it is no solution for getting data with dynamic length.

That’s it for deployment. Now, when I deploy I change YII_ENV from dev to prod and set YII_DEBUG to false and run the following three git commands

git add —all
git commit -m "my comment“
git push

Progressive Enhancement of a datadriven offline Web App

If you want to develop a data driven offline web app, you will nowadays use IndexedDB, since WebSQL is deprecated. In the native world where things are perfect you will create a download for a SQLite file, which can then be queried. In the HTML5 world you have to populate the indexed database in two steps. At first, the data must be downloaded and then this data has to be inserted via Javascript. This is somewhat cumbersome, since it lets the user wait until your highly efficient Javascript code managed to provide the offline feature. If there are a few MB of data to be inserted and there are millions of records to be created in the object stores of the database, Javascript takes its time, especially on a smartphone. I made an insert test with about 800.000 records on my macbook pro with SSD and a powerful i5 processor and it still took up to two minutes. The same procedure took 15 minutes on an android chrome browser. I did not test all different browsers on all different platforms yet. This is another problem to solve, since not every browser has the same implementation of IndexedDB. You see, while developing web apps is strategically a good choice, you start developing the data driven offline web application with a handicap. How to solve this problem? This is where progressive enhancement comes into play. Like every principle there are advocates and opponents. Despite of having more code to write to progressively enhance the user experience of a web app I can only see advantages. Concerning our problem populating the IndexedDB on the client side, why not let the user request the content from the server, until all needed data is available on the client and then query against the local database and update the page via Javascript. Web users are used to get data from the cloud be it via ajax or by a page reload. How happy are they if from a certain moment on the app works offline. They feel like on an airplane, that took off!
There are other advantages of progressive enhancement. It is important for optimizing your website for search engines (SEO). Search engine robots which visit your pages to retrieve content to their indexes are not willing to interprete Javascript. So this is another reason to let the website work without Javascript. Javascript might really enhance the user experience. But if your content can be retrieved by good old reloading the page from the server, the search engine robots have much to eat and love your website. You can make your whole database available for search engines and in this way generate many thousands of web pages in the index. This increases the number of potential visitors enormously.
Another reason to enhance a website by Javascript functionality instead of depending on it is the increasing complexity of Javascript code. Since Javascript became such a popular and important language to build websites, more and more code will be written and more and more errors are made. If this code breaks, the functionality is still available and there is more relaxation to fix it.
As a side effect is the web app usable for the 1% of users who have willingly or under constraint disabled Javascript.

Let Font Size change in Smartphone Apps

I am experiencing this for Android today but I wonder if does not apply to iPhone or web apps as well. For Android it is a big headache to give users a chance to change font sizes app wide. There are not many google results concerning this topic. How are developers trying to achieve this? The first way is to manipulate or overwrite each and every TextView of an app and set the font size according to user preferences. This assumes that you have all TextViews to manipulate. But in many cases this class is hidden at first glance. You have to find out how to write a custom layout for preferences or dialogs to name a few. Are you sure that you got each text in your application as a TextView? No hidden system font in a hidden TextView?
You can also create your own themes with its special font sizes. But font size is not the only preference witch determines a theme. Imagine you have a dark theme and a light theme. Imagine you have four different font sizes for each theme. You end up with eight themes for only two preferences. Add the next preference and you have much work to do. Since themes are static and can’t be changed at runtime it kills efficiency.
So why not set the font size at OS level? If I were disabled or could not read small fonts, I would try to solve this problem systematically. I still do not know iOS very good. Android has a preference at OS level to set the font size system wide for all apps. You don’t have to set font sizes for each app individually. Accordingly Chrome has a setting to increase the font size for all pages systematically. I am sure iOS will allow the same. Seems to be the best option for each party, users and developers!