Saturday 25 May 2013

Deploying Files to the Sailfish Emulator using the Unix rsync Command

Introduction


Note: Now adapted for Alpha 2 / Qt5. See Part 2 of this post for this technique as a script. 

 

In a previous post I showed how files could be deployed to various Harmattan and Sailfish devices.

 

For emulators and real devices we used the unix scp secure copy command.

 

Which worked. . .

 

. . . and would have worked well if Steve Jobs had not thrown  a nasty little spanner in the works.

 

My primary development Qt Host is OSX, primarily because a MacBookAir has just the right combination of portability and power that allows me develop in dead time on the train on the way to work, or like now sitting in an armchair, with Apple-Pip on my lap, a pint of ale on the floor next to my armchair, and my Linn system streaming sweet sweet music.

 

But I am not a mac-head. I point out things that are wrong with OSX, and one of those things is .DS_Store files.

 

OSX automatically litters its file system with these hidden files. These serve some internal cabalistic purpose as sanely documented here. Bill Gates's equivalents are Thumbs.db files, which are just as pesky, but at least have a vaguely amusing name.

 

While these files are harmless in themselves, they cause extra headaches when interacting with version control systems, or when replicating between heterogeneous systems.


The reference above to my Linn was not entirely spurious. For a few years I have been enjoying music steaming at CD quality and above, all stored in FLAC format on 2 NAS systems. Ever so often I sync the NASes with a backup on Butschgi my Mac Pro Tower. I use disparate sources (CDs ripped on Windows using dbPowerAmp), hi-res Albums (legally) downloaded on what ever machine i happen to be on, so after a bit of copying around and syncing between systems I like to count the number of files on each stem; but I always have the problems of those hidden .DS_Store and Thumbs.db files distorting the results .


But getting back to the Sailfish Emulator. Here we again have heterogeneous systems: our host, and the Emulator; and want to copy files from the host to the Emulator without any pesky .DS_Store files polluting our innocent Emulator.


The answer, as previously indicated is the unix command rsync.

 



Using rsync

 

First we need to install it on the Sailfish Emulator.

 

ssh -p 2223  -i ~/.ssh/mer-qt-creator-rsa  root@localhost

 

Let's just  check if it is not already installed, but I guess if you have read this far for anything than academic interest, it will not be.

 

rsync --version

 

output:

 

-bash: rsync: command not found

 

So lets find  rsync, and then install it;

 

zypper se rsync

 

zypper in rsync

 

To verify that it is now installed:

 

rsync --version

  

Back on our host  let's test the equivalent of the scp command we have used thus far: 

 

rsync -avz -e "ssh -p 2223 -i $HOME/.ssh/mer-qt-creator-rsa" flyingsheep root@localhost:/usr/lib/qt4/imports/org/

 

output:

 

building file list ... done

flyingsheep/

flyingsheep/.DS_Store

flyingsheep/abstractui/

flyingsheep/abstractui/.DS_Store

flyingsheep/abstractui/AUIButton.qml

flyingsheep/abstractui/AUIButtonColumn.qml

flyingsheep/abstractui/AUIButtonRow.qml

flyingsheep/abstractui/AUIButtonStyle.qml

flyingsheep/abstractui/AUICheckBox.qml

flyingsheep/abstractui/AUICheckBoxStyle.qml

flyingsheep/abstractui/AUIDialog.qml

flyingsheep/abstractui/AUILabel.qml

flyingsheep/abstractui/AUIMenu.qml

flyingsheep/abstractui/AUIMenuItem.qml

flyingsheep/abstractui/AUIMenuLayout.qml

flyingsheep/abstractui/AUIPage.qml

flyingsheep/abstractui/AUIPageStackWindow.qml

flyingsheep/abstractui/AUIQueryDialog.qml

flyingsheep/abstractui/AUIRadioButton.qml

flyingsheep/abstractui/AUISheet.qml

flyingsheep/abstractui/AUITabButton.qml

flyingsheep/abstractui/AUITabGroup.qml

flyingsheep/abstractui/AUIToolBarLayout.qml

flyingsheep/abstractui/AUIToolButton.qml

flyingsheep/abstractui/AUIToolIcon.qml

flyingsheep/abstractui/libabstractui.so

flyingsheep/abstractui/plugins.qmltypes

flyingsheep/abstractui/qmldir

 

sent 3543 bytes  received 4402 bytes  15890.00 bytes/sec

total size is 431766  speedup is 54.34

 

Note that the home directory on the localhost is expressed as $HOME, rather than with the tilda (that spanish squiggly thing) as used with scp. This seems to be a rsync quirk (depending on the shell) which does not allow the tilda to be resolved to the home dir. But $HOME works, is more readable (and on an OSX CH_DE keyboard I can never find the tilda anyway).

 

Now lets add the --exclude option as follows:

 

rsync -avz -e "ssh -p 2223 -i $HOME/.ssh/mer-qt-creator-rsa" flyingsheep root@localhost:/usr/lib/qt5/qml/org/ --exclude .DS_Store


output:

 

building file list ... done

flyingsheep/

flyingsheep/abstractui/

flyingsheep/abstractui/AUIButton.qml

flyingsheep/abstractui/AUIButtonColumn.qml

flyingsheep/abstractui/AUIButtonRow.qml

flyingsheep/abstractui/AUIButtonStyle.qml

flyingsheep/abstractui/AUICheckBox.qml

flyingsheep/abstractui/AUICheckBoxStyle.qml

flyingsheep/abstractui/AUIDialog.qml

flyingsheep/abstractui/AUILabel.qml

flyingsheep/abstractui/AUIMenu.qml

flyingsheep/abstractui/AUIMenuItem.qml

flyingsheep/abstractui/AUIMenuLayout.qml

flyingsheep/abstractui/AUIPage.qml

flyingsheep/abstractui/AUIPageStackWindow.qml

flyingsheep/abstractui/AUIQueryDialog.qml

flyingsheep/abstractui/AUIRadioButton.qml

flyingsheep/abstractui/AUISheet.qml

flyingsheep/abstractui/AUITabButton.qml

flyingsheep/abstractui/AUITabGroup.qml

flyingsheep/abstractui/AUIToolBarLayout.qml

flyingsheep/abstractui/AUIToolButton.qml

flyingsheep/abstractui/AUIToolIcon.qml

flyingsheep/abstractui/libabstractui.so

flyingsheep/abstractui/plugins.qmltypes

flyingsheep/abstractui/qmldir

 

sent 142471 bytes received 560 bytes 286062.00 bytes/sec

total size is 419470 speedup is 2.93

 

Note, as rsync only synchronizes changes, you may get less output this time, I purposely deleted the remote flyingsheep directory between the first and second rsync calls

 

After this we still need to set the correct unix file permissions for the files we have copied. Unfortunately also rsync (at least in my case) sets the owner and group of the files to 501 and games, rather than the rootroot I would expect.

 

rsync does have a  --chmod option, but as we need to set different options for different file types I prefer the solution below:

 

ssh -p 2223 -i ~/.ssh/mer-qt-creator-rsa root@localhost \

"cd /usr/lib/qt5/qml/org/ && \

chown -R root:root flyingsheep && \

cd flyingsheep/abstractui && \

chmod 755 *.so && \

chmod 644 *.qml &&  \

chmod 644 qmldir && \

ls -ahl && \

exit  \

; bash"

 

Here we ssh into the Emulator, and create a bash shell within which we

  • change directory to /usr/lib/qt4/imports/org/,
  • set the owner and group of the directory flyingsheep recursively to root and root,
  • change directory to /usr/lib/qt4/imports/org/flyingsheep/abstractui/,
  • apply the required file permissions,
  • list our files (so we can verify if the above commands worked,
  • and then exits ssh, back to the local shell.

As we are using the double ampersand && to stitch unix commands together, each command will only execute if the previous was successful. i.e. if there is a problem the ssh session remains open; it wil only exit and return to the host if everything works without error.

 

See UNIX tips: Learn 10 good UNIX usage habits for more on the && and other unix shell good practices.

 

I probably could combine this command with the rsync command above, but I think the result would be unreadable. A better approach would be a shell script wrapping both.

 

Conclusion

 

So using 

 

rsync -avz -e "ssh -p 2223 -i $HOME/.ssh/mer-qt-creator-rsa" flyingsheep root@localhost:/usr/lib/qt5/qml/org/ --exclude .DS_Store


and 

 

ssh -p 2223 -i ~/.ssh/mer-qt-creator-rsa root@localhost \

"cd /usr/lib/qt5/qml/org/ && \

chown -R root:root flyingsheep && \

cd flyingsheep/abstractui && \

chmod 755 *.so && \

chmod 644 *.qml &&  \

chmod 644 qmldir && \

ls -ahl && \

exit  \

; bash"

 

we can copy files to our remote device, excluding .DS_Store files. 


See Part 2 of this post for this technique as a script. 

32 comments:

  1. There are lots of information about latest technology and how to get trained in them, like UNIX Certification Courses in Chennai have spread around the web, but this is a unique one according to me. The strategy you have updated here will make me to get trained in future technologies(UNIX Course Chennai). By the way you are running a great blog. Thanks for sharing this.

    UNIX Certification Courses in Chennai | UNIX Course Chennai

    ReplyDelete
  2. At DIAC HMI training in noida is conducted by subject specialist corporate professionals. DIAC is the biggest HMI training center in Noida with high tech infrastructure and lab facilities which conducted during day time classes, weekend classes, evening batch classes and fast track training classes. Call @9310096831.

    ReplyDelete
  3. Great Article… I love to read your articles because your writing style is too good, its is very very helpful for all of us and I never get bored while reading your article because, they are becomes a more and more interesting from the starting lines until the end.

    rpa training in electronic city | rpa training in chennai

    rpa online training | selenium training in training

    ReplyDelete
  4. You made such an interesting piece to read, giving every subject enlightenment for us to gain knowledge. Thanks for sharing the such information with us
    python training in rajajinagar
    Python training in btm
    Python training in usa
    Python training in marathahalli

    ReplyDelete
  5. Thank you for allowing me to read it, welcome to the next in a recent article. And thanks for sharing the nice article, keep posting or updating news article.

    Data Science course in kalyan nagar | Data Science course in OMR
    Data Science course in chennai | Data science course in velachery
    Data science course in jaya nagar

    ReplyDelete
  6. My spouse and I love your blog and find almost all of your post’s to be just what I’m looking for.
    safety course in chennai

    ReplyDelete
  7. I know this is somewhat off-topic, but I was wondering if you knew where I could get a captcha plugin for my comment form? I’m using the same blog platform like yours, and I’m having difficulty finding one? Thanks a lot.

    AWS Interview Questions And Answers

    AWS Tutorial |Learn Amazon Web Services Tutorials |AWS Tutorial For Beginners


    AWS Online Training | Online AWS Certification Course - Gangboard

    AWS Training in Toronto| Amazon Web Services Training in Toronto, Canada

    ReplyDelete
  8. Very Informative blog thanks for sharing Searching for a SEO company in Chennai that can bring your brand to the top results page on Google.

    ReplyDelete
  9. Thanks for sharing with us, this blog was usefull to me.SEM also known as Search Engine Marketing, is focused on increasing visibility of one's newly setup or a pre-existing website. You are able to improve your site traffic SEO and SMO service. If you want digital marketing agency then go with sem services in Chennai is the best place for it.
    sem services in chennai | social media marketing companies in chennai | top 10 seo companies in chennai | top digital marketing companies in chennai | digital marketing services in chennai | best digital marketing company in chennai

    ReplyDelete
  10. Well Done Works!!! I would like to share your Blogs with My friends Circle...It's really Helpful for all..Keep Posting
    Java training in chennai | Java training in annanagar | Java training in omr | Java training in porur | Java training in tambaram | Java training in velachery

    ReplyDelete
  11. I am amzaed by the way you have explained things in this post. This post is quite interesting and i am looking forward to read more of your posts. really nice.
    Ai & Artificial Intelligence Course in Chennai
    PHP Training in Chennai
    Ethical Hacking Course in Chennai Blue Prism Training in Chennai
    UiPath Training in Chennai

    ReplyDelete
  12. Very well written blog and I always love to read blogs like these because they offer very good information to readers with very less amount of words.keep it up!!

    android training in chennai

    android online training in chennai

    android training in bangalore

    android training in hyderabad

    android Training in coimbatore

    android training

    android online training

    ReplyDelete
  13. I just recently discovered your blog and have now scrolled through the entire thing several times. I am very impressed and inspired by your skill and creativity, and your "style" is very much in line with mine. I hope you keep blogging and sharing your design idea
    hardware and networking training in chennai

    hardware and networking training in velachery

    xamarin training in chennai

    xamarin training in velachery

    ios training in chennai

    ios training in velachery

    iot training in chennai

    iot training in velachery

    ReplyDelete
  14. Thanks for your informative article,Your post helped me to understand the future and career prospects & Keep on updating your blog with such awesome article.

    java training in chennai

    java training in porur

    aws training in chennai

    aws training in porur

    python training in chennai

    python training in porur

    selenium training in chennai

    selenium training in porur

    ReplyDelete
  15. I am really enjoying reading your well written articles. It’s hard to come by experienced people about this subject, but you seem like you know what you’re talking about! Thanks.
    Java Training in Chennai

    Java Training in Velachery

    Java Training in Tambaram

    Java Training in Porur

    Java Training in Omr

    Java Training in Annanagar

    ReplyDelete
  16. This comment has been removed by the author.

    ReplyDelete
  17. This comment has been removed by the author.

    ReplyDelete
  18. This comment has been removed by the author.

    ReplyDelete
  19. I had read your blog and it has useful information. Please do posting useful infrmations it will really help us a lot and increase knowledge.

    Python Training in Chennai

    Python Training in Velachery

    Python Training in Tambaram

    Python Training in Porur

    Python Training in Omr

    Python Training in Annanagar

    ReplyDelete