Sunday 26 May 2013

Deploying Files to the Sailfish Emulator using the Unix rsync Command: Part 2

Note: As of Alpha 2: The correct install location on the Emulator for import libraries is: /usr/lib/qt5/qml/

In my last post I showed how rsync can be used to sync files to the Emulator excluding unwanted files. We also required an additional set of commands to correctly set the owner and privileges.

Since then I have been working on the Sailfish flavour of AbstractUI, and testing this with my 2 core apps. This means frequent changes to the AbstractUI qmls, and the frequent need to synchronize to the Emulator.

Getting tired of rerunning the same 2 commands; I decided to make life even easier by creating a script synchAUI2Emulator.sh:.

# This script syncs abstractui files from the localhost to the Sailfish Emulator
# and then sets owners and rights

# make this script executable with chmod 755 syncAUI2Emulator.sh
# call it with ./syncAUI2Emulator.sh

rsync -avz -e "ssh -p 2223 -i $HOME/.ssh/mer-qt-creator-rsa" org root@localhost:/usr/lib/qt5/qml/ --exclude .DS_Store \
&& \
ssh -p 2223 -i ~/.ssh/mer-qt-creator-rsa root@localhost "cd /usr/lib/qt5/qml/ && \

chown -R root:root org && \
cd org/flyingsheep/abstractui && \
chmod 755 *.so && \
chmod 644 *.qml && \
chmod 644 qmldir && \
ls -ahl && \
exit \
; bash"

The script lives in my AbstractUI project, at the same level as the org directory which contains everything I need to synchronise.

Now just one call synchronizes all changes: the output shows the files that have been synchronized, and the privileges of all files on the destination.

Saturday 25 May 2013

Deploying Files to the Sailfish Emulator using the Unix rsync Command

Introduction


Note: Now adapted for Alpha 2 / Qt5. See Part 2 of this post for this technique as a script. 

 

In a previous post I showed how files could be deployed to various Harmattan and Sailfish devices.

 

For emulators and real devices we used the unix scp secure copy command.

 

Which worked. . .

 

. . . and would have worked well if Steve Jobs had not thrown  a nasty little spanner in the works.

 

My primary development Qt Host is OSX, primarily because a MacBookAir has just the right combination of portability and power that allows me develop in dead time on the train on the way to work, or like now sitting in an armchair, with Apple-Pip on my lap, a pint of ale on the floor next to my armchair, and my Linn system streaming sweet sweet music.

 

But I am not a mac-head. I point out things that are wrong with OSX, and one of those things is .DS_Store files.

 

OSX automatically litters its file system with these hidden files. These serve some internal cabalistic purpose as sanely documented here. Bill Gates's equivalents are Thumbs.db files, which are just as pesky, but at least have a vaguely amusing name.

 

While these files are harmless in themselves, they cause extra headaches when interacting with version control systems, or when replicating between heterogeneous systems.


The reference above to my Linn was not entirely spurious. For a few years I have been enjoying music steaming at CD quality and above, all stored in FLAC format on 2 NAS systems. Ever so often I sync the NASes with a backup on Butschgi my Mac Pro Tower. I use disparate sources (CDs ripped on Windows using dbPowerAmp), hi-res Albums (legally) downloaded on what ever machine i happen to be on, so after a bit of copying around and syncing between systems I like to count the number of files on each stem; but I always have the problems of those hidden .DS_Store and Thumbs.db files distorting the results .


But getting back to the Sailfish Emulator. Here we again have heterogeneous systems: our host, and the Emulator; and want to copy files from the host to the Emulator without any pesky .DS_Store files polluting our innocent Emulator.


The answer, as previously indicated is the unix command rsync.

 


Tuesday 21 May 2013

Adding Additional Qt Packages to Sailfish SDK and Emulator

Note: 06 Nov 20 13: This post refers to Qt4 packages, thought the techniques are still relevant for the Qt5 Alpha:  Please also see the newer post:

http://flyingsheeponsailfish.blogspot.com/2013/11/deploying-additional-packages-to.html

Having posted the code of AbstractUI  to GitHub yesterday, I plan to post the code to Landed and LandedSettings: my two core apps that use AbstractUI.

However before we do that we need to address one of the first problems that you may encounter when migrating from Harmattan to Sailfish.

Your app may start up on the Sailfish Emulator, but parts of it don't do anything:

The cause?

"Missing" Qt packages on the Sailfish Alpha: Some of the packages installed as standard on Harmattan and that your app relies on, are not installed as standard on the Sailfish Alpha.

Luckily in many cases we can fix that: We simply install the packages to Sailfish.

Many things come in two halves. One example is the new Jolla smart phone just announced: Another is installing packages to Sailfish. We need to install on both the SDK and the Emulator.

In the case of Landed and its sidekick LandedSettings I had to install the packages.

  • qt-components-devel: this gives me access to the com.nokia.meego component Harmattan component set which I will use temporarily to get my apps runnable on Sailfish.
  • qt-mobility-devel: gives me access to the Qt Mobility API for things like Camera and GPS functionality.
  • sqlite-devel: for database access.

Your first step has to be to ascertain which packages are missing, and what they are called. A little bit of googling may help here (or alternatively, searching using zypper on the Emulator).

Installing to the SDK

Then our next port-of-call is the Sailfish SDK Control Center. If you have not yet stumbled over this, it is accessed by the Sailfish Icon between the Projects and Analyze icons on the icon bar on the lefthand edge of QtCreator.

On the Targets tab, click the "Manage" button. This should give you a long list of packages to install or deinstall. The rest is easy: find your desired package and click install.

Installing to the Emulator

This is initially marginally less easy, as we have to do this on the command line via SSH. We will be using the package management tool zypper:

zypper is pretty cool, as it sorts out dependencies, and will install other packages that your target package depends on.

http://en.opensuse.org/File:Zypper-cheat-sheet-1.pdf

http://en.opensuse.org/File:Zypper-cheat-sheet-2.pdf

Connect to the Emulator as described in this post.

As an example, to install the package  sqlite-devel, including other packages this depends on:

[root@SailfishEmul ~]# zypper in sqlite-devel

Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following NEW packages are going to be installed:
pkgconfig sqlite-devel

2 new packages to install.
Overall download size: 322.5 KiB. After the operation, additional 936.9 KiB will be used.
Continue? [y/n/?] (y): y
Retrieving package pkgconfig-0.27.1-1.4.i486 (1/2), 220.3 KiB (574.5 KiB unpacked)
Retrieving: pkgconfig-0.27.1-1.4.i486.rpm ........................................................[done (33.1 KiB/s)]
Retrieving package sqlite-devel-3.7.15.2-1.3.i486 (2/2), 102.2 KiB (362.4 KiB unpacked)
Retrieving: sqlite-devel-3.7.15.2-1.3.i486.rpm ....................................................[done (5.6 KiB/s)]
Installing: pkgconfig-0.27.1-1.4 ..............................................................................[done]
Installing: sqlite-devel-3.7.15.2-1.3 .........................................................................[done]

zypper also supports search, which is great if you don' t know the exact name of a package: You can find packages using the se command. e.g. to find packages with "mobility" in the name:

[root@SailfishEmul ~]# zypper se 'mobility'
Loading repository data...
Reading installed packages...

S | Name                                      | Summary                                                     | Type
--+-------------------------------------------+-------------------------------------------------------------+-----------
  | qt-mobility-debuginfo                     | Debug information for package qt-mobility                   | package
  | qt-mobility-debugsource                   | Debug sources for package qt-mobility                       | package
  | qt-mobility-devel                         | APIs for mobile device functionality - development files    | package
  | qt-mobility-examples                      | Qt Mobility examples                                        | package
  | qt-mobility-haptics-ffmemless             | Plugin which provides haptic feedback via ffmemless ioctl   | srcpackage
i | qt-mobility-haptics-ffmemless             | Plugin which provides haptic feedback via ffmemless ioctl   | package
  | qt-mobility-haptics-ffmemless-debuginfo   | Debug information for package qt-mobility-haptics-ffmemless | package
  | qt-mobility-haptics-ffmemless-debugsource | Debug sources for package qt-mobility-haptics-ffmemless     | package
i | qt-mobility-l10n                          | APIs for mobile device functionality - localization files   | package
  | qt5-qtdeclarative-import-mobility-sensors | QtQml mobility sensors import                               | package

 


Monday 20 May 2013

AbstractUI Code Posted to GitHub

I have now posted the current version(s) of the AbstractUI to GitHub

The Harmattan flavour can be found under:

https://github.com/harmattan/AbstractUI-for-Harmattan

The Sailfish Silica flavour can be found under:

https://github.com/sailfishapps/AbstractUI-for-Sailfish-Silica

Note: This library is still very much work in progress. While the Harmattan version unsurprisingly works well (the dependency on com.nokia.meego being moved form the core apps to the library), I still have some work to do on the Sailfish Silica flavor due to:

  • The conundrum of the missing components
  • Missing properties on Silica components.

This will become food for a future post ...

I also plan to add a repository with the source of my core app Landed sometime soon, once I have done some further tidying up.

Sunday 19 May 2013

Off Topic: Offline Blogger Editing

Offline Blogging

This post is a little bit off-topic, in that it is about blogging, and specifically Blog Editors; rather than the main theme of this blog which is Sailfish.
But given that the SailfsihOS is only in alpha, the communication of ideas and help amongst the nascent Sailfish community is valuable, and blogs are a vital tool to enable that. Hence this "off-topic" post.
I tend to write the first drafts of posts to this blog in dead time; often on the train to and from work; or when we have bad weather of the Föhn is blowing, either of which render flying impossible or dangerous.
This means that I prefer to build up the bulk of the content behind each article off-line.
My first attempt was with MS Word, on the basis that I have it open any way. However I found that the resulting HTML in Blogger was polluted with lots of Microsoft specific tags; and as I found myself editing the Blogger HTML more than I like, this just made life much harder than it need be.
So I next settled on TextEdit, the OSX default text editor, which actually worked well in terms of trapping content. But of course I had to format everything in the online Blogger editor.
And that is not as nice as it should be. Headings and line / paragraph spacings are a real pain. Hence the need to edit the HTML.

Deploying Import Packages to Sailfish and Harmattan Devices the Hard Way

Introduction

Note: This post will be updated for the Qt5 Alpha shortly!

In one of my recent posts I described how to create an import library / package that abstracted out imports of Harmattan and Sailfish Silica components from our core projects.

Towards the end I wrote something like "now deploy to your device" but did not say how as the article was already lengthy in danger of being long winded. Instead I snuck off and did some thing more satisfying like pouring a glass of ale.

So here is a separate article detailing how to install my abstract import package to a variety of Sailfish and Harmattan "devices".

a) The SailfishOS SDK
b) The SailfishOS Emulator
c) The Harmatttan Simulator
d) A real Harmattan Device (e.g. N9)
e) The Harmattan QEMU Emulator (will be added later)

To each of these we will add the library org.flyingsheep.abstracui, including all the .qml and library files  to the respective  "imports" directory.

Note: this article describes how to install packages manually: i.e the hard way.  While it should be possible to do this more elegantly for instance by using the "Deploy to Device" option in QtCreator, there are 2 reasons for doing it manually:
1) general nosiness and a desire to understand what is happening under the hood,
2) The "deploy to device" seems to not be properly working yet for Sailfish, as discussed in this Sailfish Developers' Mailing list thread.
 
I am using an OSX host for this article, so if you are using Windows or Linux, just change the host paths accordingly.

For some of the devices we will create symbolic links (symlinks): I have found this article showing how the equivalent can be achieved on Windows hosts: 

How to deploy an import package to the Sailfish SDK

Here the easy answer is don't!: Instead we will cheat!

My working copy of the qmls and plugins (i.e. where Qt Creator accesses them) is:
/Users/<your home dir>/SailfishProjects/abstractui/org/flyingsheep/abstractui

A proper installation on the SDK would result in:
/Users/<your home dir>/SailfishOS/mersdk/targets/SailfishOS-i486-x86/usr/lib/qt4/imports/org/flyingsheep/abstractui

But we don't want the hassle of replicating the files to the SDK every time we make a change. So let's cheat with a symlink pointing from the imports directory to the working copy. We will actually only have one copy of the files (in the working copy dir), but the operating system will "see" two directories.

We create the symlink as follows:

cd /Users/<your home dir>/SailfishOS/mersdk/targets/SailfishOS-i486-x86/usr/lib/qt4/imports

ln -s /Users/<your home dir>//SailfishProjects/abstractui/org org

Now every time we change a file in the working copy, this change will be available to the SDK without me having to do anything!

How to deploy an import package to the SailfishOS Emulator

Before we start copying, first let's connect to the device, and check what's in the imports directory /usr/lib/qt4/imports:

ssh -p 2223  -i ~/.ssh/mer-qt-creator-rsa  root@localhost

cd /usr/lib/qt4/imports

ls -l

Sailfish Emulator already has an org directory in the imports directory, so we will copy from level flying sheep (and not org)
As we can connect with the user root, we can copy direct to the right place in one step (whereas in Harmattan we have to take a multistep process)

cd /Users/<your home dir>/SailfishProjects/abstractui/org

scp -r -P 2223  -i ~/.ssh/mer-qt-creator-rsa flyingsheep root@localhost:/usr/lib/qt4/imports/org/

You should next check the file permissions: My QtCreator is hosted on OSX, and the files created have permissions that are too restrictive

So we connect to the Emulator

ssh -p 2223  -i ~/.ssh/mer-qt-creator-rsa  root@localhost

Change to the directory containing our qml and library files
/usr/lib/qt4/imports/org/flyingsheep/abstractui

Run the following 3 chmod commands:

chmod 755 *.so
chmod 644 *.qml
chmod 644 qmldir


If your host is OSX, then it will almost certainly have polluted our library with a hidden .DS_Store file in each subdirectory.
You can see these with ls -l -a
To remove these files, navigate through each subdirectory and do a rm .DS_Store

The unix command rsync has an exclude option, and can be used to deploy files as covered in this post

How to deploy an Import Package to Harmattan Simulator

One difference between the Harmattan Simulator and other devices is that the library files are binary compatible with the host rather than with a real Harmattan device. In my case this is OSX, which means that the correct library is called libabstractui.dylib instead of  libabstractui.so This can be found in /Users/<your home dir>/QTProjects/abstractui/abstractui-build-Qt_Simulator_N9-Release/

Once we have a copy of this in the working directory, we can take the same sneaky approach we took with the Sailfish SDK above: We will create an equivalent symlink:

First have a quick peak to see what is lurking in /Users/<your home dir>/QtSDK/Simulator/Qt/gcc/imports/

If all is well we can create the symlink as follows:

cd /Users/<your home dir>/QtSDK/Simulator/Qt/gcc/imports

ln -s /Users/<your home dir>QTProjects/abstractui/org org

This should result in Users/<your home dir>/QtSDK/Simulator/Qt/gcc/imports/org/flyingsheep/abstractui/

How to deploy an import package to a N9 Harmattan device

Before we start copying, first let's connect to the device, and check what's in the imports directory /usr/lib/qt4/imports:

ssh developer@192.168.2.15

enter the developer password. You will this in the "SDK Connectivity" app.

cd /usr/lib/qt4/imports

ls -l

In my case there was no "org" sub directory, so I can copy the package at the "org" level.

Unfortunately we need to take a multi-step approach as root level access is not allowed for ssh/scp (unless you do some hacking). So we will copy our package as developer to the root directory, then login and move to the right place.

Back on your host, navigate to the folder containing the N9 Harmatttan version of the org.flyingsheep.abstractui package.

cd /Users/<your home dir>//QTProjects/abstractui
 
Secure copy the directory abstract including contents with user developer:

scp -r org developer@192.168.2.15:

Connect as developer in order to create directories, and move files to proper place
ssh developer@192.168.2.15

This should connect you n the directory /home/developer. Here you should now find a subdirectory "org"

Now we need to set the unix file permissions properly

cd /home/developer/org/flyingsheep/abstractui/

chmod 644 *.qml

chmod 644 qmldir

ls -l

gives me:

total 612
-rw-r--r--    1 user     develope        58 May 19 10:38 AUIButton.qml
-rw-r--r--    1 user     develope        64 May 19 10:38 AUIButtonColumn.qml
-rw-r--r--    1 user     develope        61 May 19 10:38 AUIButtonRow.qml
…more .qml files ….
-rwxr-xr-x    1 user     develope    433033 May 19 10:38 libabstractui.so
-rw----r--    1 user     develope       696 May 19 10:38 qmldir
 
Now we need root level access
devel-su
password rootme

cd /usr/lib/qt4/imports

3) copy the abstractui directory to the imports directory

cp -r /home/developer/org/ /usr/lib/qt4/imports/org/
 

How to deploy an import package to a N9 QEMU Emulator

Coming soon ...

Conclusion

By now you should have deployed your package manually to a variety of Sailfish and Harmattan "devices".

Your next step will be to install and test apps depending on your package.  Have fun doing this!

Sunday 12 May 2013

Migrating from Harmattan to Sailfish Silica using an Abstraction Library

Introduction

Note: This post will be updated for the Qt5 Alpha shortly!

Sailfish Silica and Harmattan are related: both build on Qt  / QML: so it should be possible to port an app from Harmattan to Sailfish, or even to maintain an app that runs on both platforms from a common code base. Indeed doing so, and some of the challenges I have encountered was the trigger for this blog.

In future articles I will examine this theme from an architectural point of view, and question if a multi-platform from one code basis is actually desirable or truly possible; but in the context of this post we will assume that such an endeavor is both possible and desirable; and show how we can achieve this using an abstraction library that will remove direct dependencies on com.nokia.meego or sailfish.silica from the core code of our apps.

The Problem

Unless your app is very basic, the chances are you will have import commands such as import com.nokia.meego 1.0 in your QML code, giving you access to Harmattan components such as PageStackWindow, ToolBarLayout, and even Button, which in turn give your app the Harmattan look and feel.

Equally a native Sailfish Silica app will have imports like import Sailfish.Silica 1.0 which gives access to equivalent Silica components and look and feel.

Now if we are doing a one-way port, abandoning Harmattan in favour of Sailfish Silica, then we can just replace the import call, replace or rename some of the components, and voila we have a Sailfish app.

But if we want to maintain an update an app on both platforms then things get a little more tricky. Clearly we don't want to maintain 2 parallel projects, one importing Harmattan, and the other Sailfish. Every time we add new functionality or fix a bug we will have to remember to change both projects .... 

Equally we don't want the kludge of platform dependent IFs or IFDEFs littering our code. Our app code should concentrate on doing what it is supposed to do, and not need to worry about the platform it is running on.

To achieve this we are going to use an abstraction library.The basic concepts are well documented by Thomas Perl on his blog which you can find here

In this article I will build on Thomas's work, and address some additional problems.


Connecting to the SailfishOS VMs with WinSCP

Introduction

Note: This post will be updated for the Qt5 Alpha shortly!

Ever get fed up with navigating through unix directory structures on the command line? Having to type PWD because you have forgotten exactly where you are?
Do you need to transfer files from your localhost to your VMs?

Well if you are running in on Windows you are in luck: Meet WinSCP

For those who don't know WinSCP, it's an excellent File Transfer tool in the style of 
Norton Commander / Total Commander. It is one of the easiest ways of transferring files from Windows to Unix, and for navigating a remote Unix instance.

This is a guide to setting up WinSCP to connect to Sailfish SDK and Emulator, and in particular how to modify the RSA keys so that this is possible.


Saturday 11 May 2013

Connecting to the SailfishOS SDK and Emulator

Introduction

02 Aug 2013: Adapted for Qt5 Alpha

At some time you will want to get into the heart of the SDK and Emulators. maybe you: 

  • need to install additional packages
  • need to change some settings
  • move some files around
  • have to manually deploy or deinstall your app
  • or are just nosy, and like a small boy delight in taking things apart to find out how they work.

There are two ways to do this:

  • opening a terminal directly on the GUI of the virtual machines. I will call this VM terminal.
  • connecting via SSH from a terminal session on your local host.
In both cases you will working in a unix shell, and be able to do much the same thing. On the whole I have found it better to use the SSH from local host root because:
  • the local terminal uses my local keyboard settings, the VM terminals do not, and trying to figure out half way through a long unix command how to type US special symbols with a Swiss keyboard does not help speed up typing!
  • you can't scroll up in the VM terminal sessions - this forces you to have to pipe commands like ls to more
However there are times where SSH does not work (more on this later). In that case the VM terminal approach is a godsend.

Setting up the SailfishOS Toolkit for both SailfishOs and Harmattan:

Introduction:

Note: This post will be updated for the Qt5 Alpha shortly!

In this post I show how to setup QtCreator from the SailfishOS Toolkit so that we can develop both for Harmattan and for SailfishOS from one IDE.


Why would you want to do this?

If like me you want to port your Harmattan apps to SailfishOS, and wish to keep both versions running from a common project with a common code base, then it is very handy to be able to build run and launch our apps from one IDE in both Emulators side by side.

As a first baby-step in a migration it would be wise to get our existing apps running on the Nokia Simulator, Harmattan QEMU Emulator, and the physical device (e.g. Nokia N9) from the SailfishOS Toolkit before we even think of touching the code to port anything. This follows the basic software engineering principle of only changing one thing at at time.


Once this works, then our next challenge will be to get the same projects running on SailfishOS, and to migrate from Harmattan to Silica components…


The Harmattan to SailfishOS Blog:

Why this Blog?

02 Aug 2013: Note: Most of the posts on this blog were written for the Qt4.8.3 Sailfish Alpha: The Qt5 Alpha 2 has now been released. I am currently reviewing all posts for relevancy to the new Alpha, and will be updating / correcting them over the next few days!

Shortly after I bought my Nokia N9 many moons ago, I downloaded the Harmattan SDK and began developing a set of apps for use by paraglider pilots. My development has been on and off, sometimes centred on weekends where the weather was not good enough to fly, other times on the train in the way to work.


One of these apps has even got me out of a sticky situation on a cold and snowy mountain early this year, but that is a story for another time….(yes it does involve a big red helicopter and a long line ..)
 
Then, a few weeks ago I stumbled across SailfishOS, being developed by Jolla, a start-up staffed by many of the people who were behind the N9.
 
As the SailfishOS is the spiritual successor to Harmattan (we hope that Jolla will one day produced a physical successor to the N9), and given that both platforms are technically very similar I have been embarked on an experiment: porting my apps to SailfishOS.
 
The objective was to port my Harmattan apps to SailfishOS, and to keep them running on both platforms; the one next to dead but still physical (i.e. my phone); and the other the future, but as yet little more than a promise. Naturally I would like to do so with a high degree of common code.
 
While I was able to get my first app started on SailfishOS, almost immediately I encountered problems (let’s call them challenges 8-) ) that took a fair bit of research, creative thinking and experimentation to solve.