Happy birthday, hardware!

I was glancing at my profile recently and realized, it’s been a full six years since I built my desktop computer! I think that calls for a celebration. That’s a pretty long time in computer years.

I love my computer. I open it up about twice a year, to clean it and to adjust the fan speed for the changing seasons, and every time I do I remember why I love it so much. The internal layout is just plain lovely. Aside from needing to replace the monitor, I’ve had no failed hardware. It runs as fast and as smoothly as the day I put it together.

…which makes me so mad! How am I going to have fun buying new parts and upgrading if everything works well and suits my needs? What the crap??

But don’t worry, I still love you. Here’s to another year together, programming and blogging and music and pictures and seeing how many operating systems I can cram onto you. Which reminds me, despite having Arch Linux, Haiku, OpenBSD, and Linux Mint installed, I still have an empty partition… I think I just thought of what to give my computer for a birthday present. You can never have too many operating systems!




WebRTC

I have been waiting oh-so-long for a piece of software to save me from Skype, and today I may have found my savior!

I’ve been reading about WebRTC for years. It’s been a while now since Chrome, Firefox, and Opera announced support for WebRTC. Since then, the implementations have matured and websites have started to pop up with support for it.

The result is that, today, I used talky.io to:

  • Video chat with my family across the country
  • With no registration and instant setup
  • With clear (but occasionally just a tad laggy) video and audio
  • Using nothing but free and open source software
  • Via web browsers that are most likely already installed on many people’s computers

I’d call that a victory for freedom!




Which camera should I get?

My sister-in-law sent me a message recently asking me what I would recommend as a good camera. I love this kind of question. I would say “photography” is one of my big hobbies, but it really isn’t. Instead, my hobby is more, like, “talking about photography”.

Her needs were:

  • Control and quality (in other words, a DSLR as opposed to a point-and-shoot)
  • Shooting people, landscapes, and in low light
  • Under $600

Here was my response:

The main two camera makers are Canon and Nikon. They make a ton of cameras, and the names of them are all really confusing! Fortunately the Wikipedia makes it really easy to see all the different types and what year they came out. Just scroll to the very bottom of the page to look at the tables of camera models:

Nikon:

https://en.wikipedia.org/wiki/Comparison_of_Nikon_DSLR_cameras

Canon:

https://en.wikipedia.org/wiki/Comparison_of_Canon_EOS_digital_cameras

Sony also makes cameras. They’re a little different but still very good:

https://en.wikipedia.org/wiki/Sony_Alpha_58

I recommended the Nikon D3200, which can be bought from Amazon with the only two lenses you’ll ever need (but you can always buy more!) and a bag to carry them in for $547. You can also buy it with only one lens for less money if you like.

Now, as you’ll notice from looking at the table with all the different Nikon cameras on it, the D3200 isn’t the newest model. The D3300 already came out! So why would I recommend the D3200 instead of the D3300? Because they’re almost the exact same camera. But since the D3200 is a little older it’s also a little cheaper.

Another way to think about it is, when the D3200 came out it was a really good camera. Even though the D3300 already came out, the D3200 is still a really good camera. My point is, don’t be fooled by newer models and higher specs!

How do you choose whether to get a Canon or a Nikon camera? They are both really really good. They are the same prices and the same quality and have about the same features. Here are the differences that I know of:

  • The button layout and the menus are a little different. They both have the same basic features, but the locations of the buttons you press to be able to use those features are a little different.
  • They take different lenses. If you have a collection of Nikon lenses then you should buy a Nikon camera. If you have a collection of Canon lenses then you should buy a Canon camera. But you don’t have any lenses, so it doesn’t really matter. If you have a friend that has a bunch of lenses that you can borrow then you could buy a camera that works with them, but you probably don’t know anyone that you want to borrow lenses from anyway.
  • Canon and Nikon cameras do things a little differently, like auto white balance and auto focus. But the only people who care about these things (or can even notice them) are professionals with years of experience. I certainly don’t notice.

One more thing: The most important thing in photography is light, and by extension, the lens. What lens you use is much more important than what camera you use. The reason some cameras cost more than others is for reasons that only professional photographers care about, things like being water proof, or being able to use two SD cards at once, or having more buttons so you can switch modes more quickly. In summary, the lens is more important than the camera, and you’ll get all the features you need with pretty much any DSLR camera.

Your brother and my brother both use Canon cameras. Your aunt and I both use Nikon cameras. My coworker uses a Sony camera. They’re all good.




Setting a background image when using a WM

In order to set the background image for my WM (spectrwm <3) I use hsetroot and point it to ~/.taustakuva [taustakuva means wallpaper in Finnish <3], a symbolic link in my home directory:

  1. $ ln -sf path_to_your_picture ~/.taustakuva
  2. Then I simply add this line to ~/.xinitrc: hsetroot -center ~/.taustakuva

For eyecandy I use compton, a lightweight and advanced X compositor forked out of xcompmgr, which gives a nice transparency effect (among others) to urxvt:
1. Add to ~/.xinitrc: compton -o1 -b
2. If using urxvt (rxvt-unicode-256colors) don’t forget to set either:

  • URxvt*background : [95]#your_color_code_in_hexadecimal
  • URxvt.transparent : true

in ~/.Xdefaults according to your taste.
You will end with something like this:

 

 

Cheers.




Arch rocks! (fanart wallpaper)

Just a fanart wallp I made some time ago that I’m using again as my GRUB background:

 

Arch rocks!
Real size: 2048×1152

 

To set a picture as GRUB’s background open /etc/default/grub and follow these steps:

  1. Uncomment and point GRUB_BACKGROUND= to where your picture lies, i.e. /boot/grub/Arch_Rocks.png. Bear in mind that the directory where your picture reside must be mounted at the time GRUB is loaded so best place usually is /boot/{whatever};
  2. Generate new GRUB menu: # grub-mkconfig -o /boot/grub/grub.cfg

 




Xorg: Using the US International (altgr-intl variant) Keyboard Layout

Source: all praise and glory goes to Shinobu’s Blog

Since I switched to spectrwm I found that I had to explicitly tell X to use the keyboard layout I find most useful.

Let’s say you’re using the US Intl. keyboard layout (the only sane layout if you ask me) and you need to enter special symbols and characters, the altgr-intl layout – aka French/English layout on RH and its siblings distros – is the way to go then as “all of the non-US keys are all hidden behind a single key: the right Alt key.”.

So in order to instruct X to use this useful keyboard layout you could:
1. Add the corresponding section to Xorg’s keyboard settings by creating the appropriate file in /etc/X11/xorg.conf.d – not really recommended as it is somewhat awkward nowadays and the least you fiddle with your admin account the less you probably end up messing it;
2. Just add setxkbmap -rules evdev -model evdev -layout us -variant altgr-int to your ~/.xinitrc file (or any other init script parsed by your WM);

Now writing things like ‘Yo preparo la salsa para los ñoquis, te vas a chupar los dedos” o “Hyvä päivä Suomessa!” is easy again :)

-Martin




The Inofficial Arch Games Collection

Tuxracer screenshot, by linuxblog.dk

Of course there are hundreds (if not thousands) of games in the Arch repositories, including popular classics like The Battle Of Wesnoth, Nexuiz, or Flightgear. But there are some games that go even better with Arch’s KISS philosophy, and that COULD be included with Arch, if Arch came on a big DVD with games included and so on, so, long story short, here’s The Inofficial  Arch Games Collection” ;)

  • Pysolfc – The inevitable collection of solitaire card games, plus a very nice Mahjong
  • Extreme Tux Racer – The classic Tux downhill racer, still loads of fun
  • Pychess - Beginner- friendly chess program with a helpful “Hints” and “Spy” mode. Pychess has its own engine but can also use gnuchess or any other Xboard- compatible engine.
  • Ltris - Lightweight implementation of the popular classic “Tetris” with many features
  • DoomRL LQ - simple and beginner- friendly (at least on level “too young to die”) rogue- like with tile graphics. The LQ version is only 5 MB vs. 80+ MB for the “full” version and has midi music instead of MP3s
  • FreeCiv – Free clone of Civilization 2, single- or multiplayer
  • Dopewars - classic … erm … business simulation that can be played in GTK or on the command line (option -t)
  • Lbreakout - Nice breakout game with a lot of options
  • GzDoom - Nice OpenGL DOOM implementation with pretty good graphics, keyboard playable !! You need additional game data, e.g. FreeDoom / Blasphemer from the AUR or download Harmony from their web page
  • Assault Cube - Lightweight (~50MB) single- / multiplayer FPS, also playable with the touchpad (if you have to)Singularity - Unique (round- based) game about an artificial intelligence called the Singularity
  • Micropolis - The original “Sim City” game, open- sourced and adapted for the OLPCproject
  • OpenTTD - Free clone of the popular classic “Transport Tycoon”
  • Oolite - OpenGL remake of the classic Elite / Frontier game, including the classic vector graphics
  • GL-117 – simple but fun (combat) fligh sim

This is just a small collection of my favorites. More interesting games can be found in the wiki under Netbook Games and List of Applications / Games. All these will of course work in other Arch- based distributions like Manjaro, Archbang or Bridge, too.




Creating Apps In Kivy: The Book

I apologize for my near-silence over the past few months. I’ve packed house and moved twice in that time, started a new job, and, relevant to this post, written a book.

My Creating an application in Kivy blog series has been by far the most popular set of posts on this blog. It garners about 200 views per day, and has been doing so for nearly a year. This taught me a very important lesson: posts with controversial titles, (notably this one, but basically anything that has hit hacker news) are much less valuable than posts with meaningful long-term content. Getting a few thousand hits in one day is pretty satisfying, but when most of your “readers” simply comment on the post without having read the article, that hit-count becomes meaningless.

“Creating an application in Kivy”, on the other hand, has had sixteen thousand visits to part 1, and nearly three thousand of those have made their way all the way to part 9. I could definitely achieve more hits per day by writing a new minimum-content controversial article every day, but these visitors keep coming with no effort on my part. This is lucky, as the focus on writing solid content for my book has taken time away from writing solid content for this blog.

And with that, I’d like to announce that Creating Apps In Kivy is now available. This project started when the aforementioned tutorial caught the eye of an O’Reilly editor. They helped me put together a great book that I am proud of and confident you will enjoy.

Creating Apps in Kivy covers many of the topics discussed in my tutorial, but in more depth and with much more polish. Like the tutorial, it takes you from “new to Kivy” to creating and deploying a complete application to your Android or iOS device. It’s a fun, short read and I’m proud to present it to you today. If you liked the tutorial, buying my book would be a great way to support my work!




Have you ever been afraid to get out of bed?

In my past writings about mental health, I’ve mostly discussed depression and suicide. I’ve been meaning to add ‘anxiety’ to that list of topics, but have been waiting until I could write about it from the first person. One of the peculiar things about my personal struggles with mental illness is that when I’m feeling well, it’s really hard to remember what it’s like to be feeling poorly. The inverse is also true, which is why it’s possible to feel suicidal after just a few hours of depression; I can’t imagine what it felt like yesterday when I’d been feeling fine for months on end.

I’ve got a great handle on what I need to do in my life to deal with, prevent, and accommodate for my depressive, suicidal, and bipolar tendencies. I’ve also made great strides in reducing anxiety levels, but, for me, anxiety is a more difficult beast to tame. The following was written a few days ago when I was in the middle of a high-anxiety experience (not an anxiety attack). I’ve edited it a bit for grammar (typing on a phone is a horrible experience), but it’s mostly straight from the heart. I hope it illustrates what anxiety feels like to those who have not experienced it, so you can support those in your life who are suffering. I would also like it to demonstrate to those who may not be aware that they are dealing with anxiety that their symptoms are treatable, and perhaps you will seek further help. And, as always, I want those of you who deal with anxiety on a regular basis to remember that you are not alone.

From Inside Anxiety

Let me tell you what anxiety feels like. It feels like being on a small rock amidst a boiling vat of lava. I have to type this on a phone because the laptop in the other room is no less inaccessible than if I had to jump a giant ravine. Anxiety is fear, pure simple fear. But it’s not normal fear; anxiety is fear of the normal.

Have you ever experienced an adrenaline rush? Some threatening situation: the grill of an oncoming bus, falling down the stairs, being approached by a thug with knife in hand, slipping on ice, a shove from behind, a car accident?

Anxiety is that same feeling, but all the time. Every step is a slip, every car is going to hit you, all your friends and co-workers are thugs with knives. The worst part is that the brain can see that these sensations are not true. I’m not imagining real lava around the bed. I can see the perfectly safe carpet. But my legs feel like numb jello and I cannot step onto it. I’m not hallucinating that all the people outside my street want to kill me. I’m afraid of them even though I know they have no interest in me at all. I know in my mind that my office is full of people I can trust and rely on, yet my body is reacting as though I will be entering a war zone, every sense alert, every muscle tensed against the next attack.

Once upon a time, I lived the majority of my life’s moments in such a constant state of heightened awareness. Fear. It was exhausting. I am lucky that days like this are now rare. I am lucky that today, my anxiety is not accompanied by depression (it’s hard to feel worthy when you spend all your time hiding in bed.).
Some people look for adrenaline rushes outside their normal state of being. They sky dive, they become immersed in video games or movie thrillers, they join extreme sports. While they do this, I look for quiet rooms, I meditate, I practice yoga. Anything to calm the blood down. I will experience the same thrill climbing into the shower this morning that you feel jumping out of that plane. I will feel the same fear walking into my office today that you feel as you step onto a public stage.

And here’s the kicker: nobody thinks I’m brave. My walk to work today will mirror an Indian Jones escape sequence or Aladdin’s carpet ride out of the Cave Of Wonders. I’ll walk into my office and be surrounded by people who do that kind of thing every day. Anxiety is living your life inside a thriller movie. The music is always loud, rushing your heart beat. Something awful is always going to happen around the next corner. This movie doesn’t end and you can’t walk out of the theater.

The world is going to end if I don’t get out of bed and into the shower. How much is a world worth?

Dealing With Anxiety

I have had less success in defeating anxiety than I have had with depression. There are a few types of treatments to deal with it, and you should consult with a medical professional to work out what the best one is for you. The two things that have had the greatest success in taming (if not controlling, yet) my anxiety have been yoga/meditation, and exposure therapy. The latter isn’t as bad as it sounds, although like many cures, it’s not always pleasant. The heart of exposure therapy is, as you can expect from the name, placing yourself in situations that cause anxiety. However, there are a few caveats. You have to place yourself in such situations until you become comfortable with them. If you exit an anxiety-inducing situation while you still feel afraid, you will have reinforced that this is a fearful situation, and that “fight or flight” is an appropriate response.

Instead, you need to pick situations that only cause a small amount of anxiety. A good way to do this is to make a list of as many situations as you can think of that cause anxiety, and then rank them in order of how frightening they are. Over the course of a year or so, put yourself into the easiest situations and work your way up the chain. It’s slow progress, but it’s measurable, and the feedback you give yourself from one encounter can help you refine your technique in a later one.

I’ve been utilizing this technique for three years. As illustrated above, I still have days with a lot of anxiety. However, I lead a functional life, now. I used to work from home, I now work for a high-profile company and I’m doing well at it. I used to avoid all social situations, I now regularly go out with friends or even strangers and generally have a good time. I used to be especially afraid to interact with single women; I’m now in a stable, loving relationship. There are still a few things on my list, such as presenting at conferences and dealing better with conflict, but for the most part my coping skills are better than many people who are considered not to have psychological disorders. It wasn’t easy, but it was worth it.




OpenBSD

I had a new experience last week: I installed OpenBSD. And wow, what an experience it was!

As a fan of trying new operating systems, I’ve been wanting to “learn BSD” for many years now. I had decided a while back to try out FreeBSD because it has the largest user base, and assumed it would be the “best BSD experience” for me. Well, my experience with FreeBSD was that documentation was sorely lacking (despite hearing the contrary) and the things I was most interested in seemed to be in a state of flux, namely package management and compilers. That and the fact that there doesn’t seem to be “one right way” to do almost anything in FreeBSD caused me to never really understand what I was doing, no matter how hard I tried. Or maybe I’m just an idiot. I don’t know.

Well, OpenBSD 5.4 just came out, and after reading a comment on OSNews from a fan absolutely gushing over it, I became really excited to try it out.

Now, I only knew a few things about OpenBSD:

  • It’s secure. Like, VERY secure.
  • The documentation is vast and superb.
  • It’s not very popular, even in the list of not-very-popular operating systems.
  • The mascot is cute.

I downloaded the latest release and read a bit of the installation guide. I was very impressed with the documentation, but not nearly as impressed as I was going to be after installing everything. One thing in particular stood out to me in the installation guide: the advice to install OpenBSD on an empty hard drive if it’s your first time. Pshhh, I’ve installed a million operating systems. How hard could it be?

Well, it wasn’t hard, but it was very different from what I was used to. At one point near the end of the installation after configuring the partitions, I had no idea whether or not all my personal files were still on my hard drive. After wiping my sweaty palms together and rebooting my computer, it turns out I’d done everything correctly and didn’t erase anything. Well, I guess I should say OpenBSD did everything correctly. For all the power it provides, the installer really is pretty automatic and smart. Anyway, as a responsible computer user I of course have a recent backup too, but it’s still nice to not have to need it.

Success!

Success!

And it’s at this point that the documentation really impressed me. I finish the installation and reboot. I login and it tells me I have a new email that I can read with the “mail” command. The email gives a welcome and a ton of information, including the suggestion to read “man afterboot”. That man page gives even more instructions on how to do things people often want to do after an installation. Every command has a manual. Ever configuration file has a manual. The documentation just kept going and going! And it’s all both high quality and current.

I installed a few applications with ease using the built in package manager. Although the X server is not “part of” the operating system, it was included in the install and was automatically configured.

I started X and had to laugh. OpenBSD is very much UNIX, and if ever I saw a GUI that screamed “UNIX” it’s the default in OpenBSD.

I can almost TASTE the 1970s.

I can almost TASTE the 1970s.

I decided to keep it, just because it makes me feel all hardcore UNIXy.

Also, OpenBSD feels fast. I don’t know if it’s just in my head or what, but the Internet connection in particular just feels so smooth compared to Arch Linux.

And that was pretty much it. The Internet, video (including 3D Radeon drivers), and audio were all configured automatically. I’m incredibly impressed with what I’ve seen. I haven’t decided where to go with OpenBSD from here, besides just “learn it more”, which I look forward to doing.

…posted from OpenBSD!




I’m Not Selling Out The Gittip Dream

A couple weeks ago, I signed a contract with O’Reilly to publish a book inspired by my extremely popular Creating An Application In Kivy blog series. On Monday, I start work at Facebook, a full-time salaried position that includes a substantial raise over my last position.

Signing these two contracts, two very traditional ways of obtaining money in exchange for services, is in stark contrast to the excitement I have held for the Gittip project. I marketed (some might say ‘begged’) for Gittip contributions in return for the service I performed in writing the Kivy articles. Gittip provided; I really didn’t expect to make more than a couple dollars per week, but my current income from Gittip is over $11 per week. I’m touched and grateful for these donations, and I don’t feel I deserve them at all, considering that I’ll be spending most of my time on my new book and job for the next few months.

It would be easy to argue that Gittip has failed. If I had somehow made more money off Gittip, I might have chosen to publish a book under a creative commons license or through other platforms, rather than signing a traditional book contract. If I was making so much money off Gittip that I could have laughed off Facebook’s salary and stayed out of the horrifically over-priced silicon valley, the open source world (and Facebook’s competitors!) would certainly have benefited.

However, there is a better way to look at this. I have committed (to myself) to contribute a substantial portion of the royalties from my new book back to the Kivy team. I will naturally use Gittip to distribute these funds. I am also planning to increase my funding of other projects out of my Facebook salary, and I’ll be using Gittip to fund the Arch Linux development team (if they ever set one up on Gittip) out of the income I already make from Arch Linux Schwag.

I believe one of the biggest problems Gittip has to solve is not increasing it’s member base, but increasing the amount of money being injected into the system. If small tips are just moving from one open source developer to another and back again (I’ve gotten into the unfortunate habit of regifting everything I make plus a few dollars), the money really has no value. It is a small gesture, but it’s not something that can be turned into food on the table if everyone just turns around and gives it away. I would definitely like to live a life, someday, where my writing and coding activities are exciting enough to the world that my entire salary comes from Gittip. However, I think the best thing I can do for the tool right now is to put money in, rather than take it out.

So no, Gittip has not failed me. It is succeeding in a different way from what I originally anticipated. Nor am I selling out. I am excited and passionate about my current and future prospects. Gittip was designed to allow content creators to pursue their passions. I’m able to do that within the traditional framework. I’m very lucky to be able to express my gratitude for my current life in the form of microdonations to other developers.




A postgres backup script for a snapshot environment


#!/bin/bash

# Backup
# Usage: backup-pg.sh
#

# If you have another method for obtaining your filer's ip
# use that instead to assign this variable
FILERIP=`mount | grep db01 | cut -d':' -f1`

# username associated with the key used
UN="snappy"
# location of the ssh key used for above user
SNAPKEY=/opt/bbcollab/voice/postgres.stuff/snapkey

# If you have another method for obtaining the volume name or archive
# Edit the two lower variables; your ARCHIVE name should have a grepable line
VOLUME=`mount | grep db01 | cut -d':' -f2 | sed 's#/vol/##g' | sed 's#/db01.*##g'`
COMNAME="backup_pgdata"
ARCHIVE="${COMNAME}-${HOST}-${TODAY}"

# location of the .snapshot directory
SSDIR=/var/voice/postgres/.snapshot

# retention in snapshot count
RETEN=4

function get_oldest_snapshot_by_filesystem
{
ss=`ls -lct $SSDIR | grep -v "total" | head -1 | awk '{print $9}'`
echo $ss
}

function get_oldest_snapshot_by_netapp
{
# looking for archives with ${COMNAME}
cur_ss=`ssh -i ${SNAPKEY} -o StrictHostKeyChecking=no ${UN}@${FILERIP} "snap list ${VOLUME}" | grep "${COMNAME}" | awk '{print $10}' | grep -v "^$" | tail -1`
echo $cur_ss
}

function remove_oldest
{
ssh -i ${SNAPKEY} -o StrictHostKeyChecking=no ${UN}@${FILERIP} "snap list ${VOLUME}" | grep "${COMNAME}" > /var/tmp/ss.lst
ss_count=`wc -l /var/tmp/ss.lst | awk '{print $1}'`
if [[ "$ss_count" -gt $RETEN ]]
then
obf=`get_oldest_snapshot_by_filesystem`
obn=`get_oldest_snapshot_by_netapp`
if [[ "$obf" == "$obn" ]]
then
ssh -i ${SNAPKEY} -o StrictHostKeyChecking=no ${UN}@${FILERIP} "snap delete ${VOLUME} $obf"
fi
else
echo "none to remove"
fi
}

function usage
{
echo ""
echo "### $0 Usage ###"
echo ""
echo "Call script like:"
echo ""
echo " $0 ()"
echo " (cron uses -a)"
echo ""
echo "Options:"
echo " -a #### Create a new snapshot and delete the oldest"
echo " -c #### Only create a new snapshot"
echo " -l #### List the current snapshots"
echo " -d Delete snapshot (regardless of age or retention)"
echo " -q #### Delete the oldest snapshot (will not delete if you have fewer snapshots than $RETEN)"
echo ""
}

if [[ $# > 0 ]]
then
case $1 in
-a) psql -U postgres -h localhost -c "SELECT pg_start_backup('${ARCHIVE}');" template1
ssh -i ${SNAPKEY} -o StrictHostKeyChecking=no ${UN}@${FILERIP} "snap create ${VOLUME} ${ARCHIVE}"
psql -U postgres -h localhost -c "SELECT pg_stop_backup();" template1
remove_oldest
exit
;;
-c) psql -U postgres -h localhost -c "SELECT pg_start_backup('${ARCHIVE}');" template1
ssh -i ${SNAPKEY} -o StrictHostKeyChecking=no ${UN}@${FILERIP} "snap create ${VOLUME} ${ARCHIVE}"
psql -U postgres -h localhost -c "SELECT pg_stop_backup();" template1
exit
;;
-l) ssh -i ${SNAPKEY} -o StrictHostKeyChecking=no ${UN}@${FILERIP} "snap list ${VOLUME}"
exit
;;
-d) ssh -i ${SNAPKEY} -o StrictHostKeyChecking=no ${UN}@${FILERIP} "snap delete ${VOLUME} $2"
exit
;;
-q) remove_oldest
exit
;;
*) usage
exit
;;
esac
else
usage
exit
fi




http://usalug.com/phpBB3//viewtopic.php?f=15&t=8086


#!/bin/bash
# Author: Josh Bailey
# Email: jbsnake gmail.com
# A function/subroutine library
# A lot of these functions and subroutines I use
# in all my scripts, some I have only used once.
# This is a lot of hard work right here, so please
# don't get tempted by saying you did it yourself :)
# If you use something here (which you ofcourse are free to do)
# Please atleast give a link to www.usalug.org or www.bashscripts.org
# You don't have to mention me, just mention where you found such awesome code ;)

function notAForwardSlash
{
if [[ ${1} != '/' ]]
then
return 0
else
return 1
fi
}

function notADot
{
if [[ ${1} != '.' ]]
then
return 0
else
return 1
fi
}

function getFileName
{

# call this function like
# filename_variable=`getFileName /some/path/with/a/file.name`

local STRING=$1
local LENGTH=${#STRING}
local n=0
local FileName=""
for ((n=0;n <= $LENGTH; n++))
do
local CHAR=${STRING:$n:1}
if notAForwardSlash $CHAR
then
FileName=$FileName$CHAR
else
FileName=""
fi
done
echo "FileName"
}

function getExtension
{

# call this like
# extension_variable=`getExtension /path/to/some/file.txt`
# extension_variable would be "txt" for that example
# you can also just pass it a filename
# extension_variable=`getExtension somefile.lst`
# extention_variable would be "lst" for that example
# but if you pass it a filename or path without a .
# it will return exactly what you pass it
# extension_variable=`getExtension /path/to/my/script`
# extension_variable would be "/path/to/my/script"

local STRING="${1}"
local LENGTH="${#STRING}"
local n=0
local Extension=""

for ((n=0; n <= $LENGTH; n++))
do
local CHAR=${STRING:$n:1}
if notADot $CHAR
then
Extension=$Extension$CHAR
else
Extension=""
fi
done
echo "$Extension"
}

function getPath
{

# call this like
# chop_path=`getPath /path/to/something.file`
# chop_path would be populated with "/path/to/"

# function to get only the path from a path/filename combo
# first get the filename from $1 which is a path/filename combo
local just_filename=`getFileName "$1"`

# set the variable containing the path
local newPath=${1:0:(${#1}-${#just_filename})}
echo "$newPath"
}

function inStr
{

# call this function like
# instr_variable=`inStr "=" "1=2"`
# since the position of = is 1 (i.e. 0 1 2 )
# instr_variable would equal 1

local searchFor="$1";
local lenOf=${#searchFor};
local searchIn="$2";
for (( i=0; i /dev/null< /dev/null
if [[ $? -eq 1 ]]
then
return 1
else
touch ${1}/bob.yea\ do0d 2> /dev/null
if [[ $? -eq 1 ]]
then
rm ${1}/bob.yea\ do0d 2> /dev/null
return 1
else
rm ${1}/bob.yea\do0d 2> /dev/null
return 0
fi

fi
}

function trimspaces
{

# call this like
# new_string=`trimspaces "some random text" " "`
# new_string variable would be "somerandomtext"
# new_string=`trimspaces "some_random_text" "_"`
# new_string variable would be "somerandomtest"
# new_string=`trimspaces "some random_text"
# new_string variable would be "somerandom_text" (default, if nothing is passed, to remove are spaces)

local NewString=""
local STRING="$1"
local rmChar="$2"
local CHAR=""
local NewString=""

if [[ $rmChar = "" ]]
then
rmChar=' '
fi
for ((i=0; i<${#STRING}; i++))
do
CHAR=${STRING:$i:1}
if [[ $CHAR != $rmChar ]]
then
NewString=$NewString$CHAR
fi
done
echo "${NewString}"
}

function fileExist
{

# call this like
# if [[ fileExist "/path/to/some/file.name" ]]
# then
# yes it does
# else
# no it doesn't
# fi

if [[ -a "$1" ]]
then
return 0
else
return 1
fi
}

function isNumeric
{

# call this like
# if [[ isNumeric ]]
# then
# yes it is
# else
# no it's not
# fi

local Numeric='y'
local len=`echo ${#1}`
local val=""
local retval=""

for ((i=0; i /dev/null
retval=$?
if [[ $retval -eq 1 ]]
then
if [[ $i -eq 0 && $val = '-' ]]
then
num="negative"
else
Numeric='n'
break
fi
fi
done
if [[ $Numeric = 'y' ]]
then
return 0
else
return 1
fi
}

function isAlpha
{

# call this like
# if [[ isAlpha ]]
# then
# yes it is
# else
# no it's not
# fi

local Alpha='y'
local len=`echo ${#1}`
local val=""
local retval=""

for ((i=0; i /dev/null
retval=$?
if [[ $retval -eq 1 ]]
then
Alpha='n'
break
fi
done
if [[ $Alpha = 'y' ]]
then
return 0
else
return 1
fi
}

function isAlphaNum
{

# call this like
# if [[ isAlphNum ]]
# then
# yes it is
# else
# no it's not
# fi

local AlphaNum='y'
local len=`echo ${#1}`
local val=""
local retval=""

for ((i=0; i /dev/null
retval=$?
if [[ $retval -eq 1 ]]
then
echo $val | grep ^[0-9] > /dev/null
retval=$?
if [[ $retval -eq 1 ]]
then
AlphaNum='n'
break
fi
fi
done
if [[ $AlphaNum = 'y' ]]
then
return 0
else
return 1
fi
}

function initialCaps()
{

# call this like
# new_string=`initialCaps "john doe"`
# new_string variable would be "John Doe"

local Cap=""
local Info=""
local Word=""
local string="${1}"
local cut=""

while [ ${#1} != 0 ]
do
Cap=`echo ${string:0:1} | tr [:lower:] [:upper:]`
Info=`echo -n $Cap; echo ${string:1:${#string}} | tr [:upper:] [:lower:]`
Info=${Info}" "
Word=${Word}${Info}
shift 1
done
cut=${#Word}-1
Word="${Word:0:${cut}}"
echo "${Word}"
}

######## Music file format stuff #################################

function get_ogg_info
{

# call this like
# ogg_info_string=`get_ogg_info "/path/to/file.ogg"`
# ofcourse the string would have to be parsed
# it is pipe | delimited
# in order artist, title, album, genre, date, and track number
# inStr function needed; vorbiscomment (comes with oggenc)
local turn=""
local index=0
local item=""
local cartist=""
local ctitle=""
local calbum=""
local cgenre=""
local cdate=""
local ctracknumber=""

vorbiscomment -l "$1" > info.lst
for turn in artist title album genre date tracknumber
do
tmp_comment=`grep -i "$turn" info.lst`
item=`inStr "=" "$tmp_comment"`
comment=${tmp_comment:${item}+1}
((index++))
case $index in
1) cartist="$comment";
;;
2) ctitle="$comment";
;;
3) calbum="$comment";
;;
4) cgenre="$comment";
;;
5) cdate="$comment";
;;
6) ctracknumber="$comment";
;;
esac
done
info="${cartist}|${ctitle}|${calbum}|${cgenre}|${cdate}|${ctracknumber}"
echo "${info}"
rm -f info.lst
}

function encode_flac2mp3
{

# call this like
# encode_flac2mp3 /path/to/source/file.flac /path/to/destination
# needs: getFileName function; flac encoder/decoder; lame

local old_file="${1}"
local new_dir="${2}"
local short_filename=`getFileName "${old_file}"`
local new_file="${short_filename:0:${#short_filename}-5}.mp3"

flac -d -o - "${old_file}" | lame -b 320 -h - > "${new_dir}/${new_file}"
}

function encode_flac2ogg
{

# call this like
# encode_flac2ogg /path/to/source/file.flac /path/to/destination
# needs: getFileName function; flac encoder/decoder; oggenc

local old_file="${1}"
local new_dir="${2}"
local short_filename=`getFileName "${old_file}"`
local new_file="${short_filename:0:${#short_filename}-5}.ogg"
###### get artist and album before release #########
# flac -d -o - "${old_file}" | oggenc -a "$artist" -l "$album" -t "${title}" - -o "${new_dir}/${new_file}"
####################################################
local title="${short_filename:0:${#short_filename}-4}"

flac -d -o - "${old_file}" | oggenc -t "${title}" - -o "${new_dir}/${new_file}"
}

function encode_ogg2mp3
{

# call this like
# encode_ogg2mp3 /path/to/source/file.flac /path/to/destination
# needs: getFileName function; oggdec; lame

local old_file="${1}"
local new_dir="${2}"
local short_filename=`getFileName "${old_file}"`
local new_file="${short_filename:0:${#short_filename}-4}.mp3"
local info_string=`get_ogg_info "$old_file"`
local cartist=`cut -d| -f1 ${info_string}`
local ctitle=`cut -d| -f2 ${info_string}`
local calbum=`cut -d| -f3 ${info_string}`
local cgenre=`cut -d| -f4 ${info_string}`
local cdate=`cut -d| -f5 ${info_string}`
local ctracknumber=`cut -d| -f6 ${info_string}`

oggdec "${old_file}" -o - | lame -b 320 --tt "$ctitle" --ta "$cartist" --tl "$calbum" --ty $cdate --tn $ctracknumber --tg "$cgenre" -h - > "${new_dir}/${new_file}"
sleep .5
}

####################################################################################

############################ gpg stuff #############################################

function list_private_keys
{

# call this like
# list_private_keys
# it just clears the screen then
# pipes the private items on your keyring to less

clear;
echo ":: Listing private keys";
gpg --list-secret-keys | less;
echo ":: Done";
}

function list_public_keys
{

# call this like
# list_public_keys
# it just clears the screen then
# pipes the public imported keys on
# your keyring to less

clear;
echo ":: Listing public keys";
gpg --list-keys | less;
echo ":: Done";
}

function decrypt_any_file
{

# call this like
# decrypt_any_file /path/to/some/encrypted/file.asc
# it clears the screen then prompts you for your passphrase
# then decrypts the encrypted file into another file

local filename="$1";
local person=`whoami`;
clear;
if [[ ${filename:${#filename}-11:11} -eq "-encrypted" ]]
then
local newfilename="${filename:0:${#filename}-${#person}-12}";
gpg -d --output "$newfilename" "$filename";
echo ":: Decrypted $filename";
echo ":: Saved as $newfilename";
else
gpg -d --output "$filename.decrypted" "$filename";
echo ":: Decrypted $filename";
echo ":: Saved as $filename.decrypted";
fi
}

function encrypt_any_file
{

# call this like
# encrypt_any_file /path/to/file/to/encrypt.asc "who it's encrypted for"
# it clears the screen then encrypts the file you specified
# saving the encrypted file to have the persons name and "-encrypted" at the end

local filename="$1";
local person="$2";
clear;
gpg -sea --output "$filename"."$person"-encrypted --recipient "$person";
echo ":: File $filename encrypted";
echo ":: Encrypted file saved as $filename.$person-encrypted";
}

function export_public_key
{

# call this like
# export_public_key "Key Identifier"
# generates a key pair and saves the public key as "Key Identifier.pub"

local ID="$1";
gpg -ao $ID.pub --export $ID;
echo ":: Public Key for $ID has been generated"
echo ":: File is saved as $ID.pub"
}

function import_public_directory
{

# call this like
# import_public_directory /path/to/directory/full/of/public/keys
# changes directory to the public key directory and imports every file there
# then goes back to the directory you started in

local DIR=$1;
cd "$DIR";
for filename in *
do
gpg --import "$filename";
done;
cd ~-;
}

function create_encrypted_ascii_file
{

# call this like
# create_encrypted_ascii_file /path/to/location/for/future/message.asc "Who to encrypt for"
# opens a vi session (note you can make the editor whatever you want by changing the variable)
# once you close the editor, the file you just edited is encrypted
# then it's echo'd to the console for easy copy and pasting

local editor="vi"
local file="${1}";
local person="${2}";
${editor} "${file}";
gpg -sea --output "${file}.${person}-encrypted" --recipient "${person}" "${file}";
clear;
echo ":: Encrypted file created";
echo ":: File saved as ${file}.${person}-encrypted";
echo ":: Contents of file shown below";
echo -e "\n\n";
cat "${file}.${person}-encrypted"
}

function import_public_key
{

# call this like
# import_public_key
# it prompts you for the key-file you wish to import

read -p "Please type the path of the public key you wish to import: " file
if [[ -e "$file" ]]
then
gpg --import "$file"
else
echo "${file}: doesn't exist, or you typed the wrong path!"
exit
fi
}

function create_user_list
{

# call this like
# create_user_list
# makes a list of users on the system
# it grabs the contents of /etc/passwd
# then removes the entry for root
# and saves the list as users.list

cat /etc/passwd | cut -d: -f1 > pre.list;
cat pre.list | grep -v "root" > users.list;
rm pre.list;
}
#####################################################################################3

function app_up
{

# call this like
# app_up /path/to/file/append.from /path/to/file/append.to
# puts data from a file at the top of another file instead of the bottom

if [[ ! -e ${2} ]]
then
touch ${2}
fi

${1} > temp
cat ${2} >> temp
rm -f ${2}
mv temp ${2}
}

function createROMISO
{

# call this like
# createROMISO /dev/cdrom /path/to/new/rom.iso

# the command that makes an ISO from a cd/dvd
# $1 should be the device path (/dev/cdrom)
# $2 should be the iso path and filename
dd if=$1 of=$2 bs=32k
}

function createDIRISO
{

# call this like
# createDIRISO /path/to/new/rom.iso /path/to/directory/you/want/to/backup

# the command that makes an ISO from a directory
# $1 should be the iso path and filename
# $2 should be the directory to backup
mkisofs -o $1 $2
}

function msg
{

# call this like
# msg "Title of Message Box"
# or
# msg "Title of Message Box" $height $width

# function for making a messagebox
# if it has less than two arguments
if [[ $# < 2 ]]
then
# use auto-size for the messagebox
dialog --msgbox "$1" 0 0
else
# use specified height and width
dialog --msgbox "$1" $2 $3
fi
clear
}




Shooting An Iceray

So I am working on learning a few things, all at the same time. The coolest thing is Go, the “programming environment that makes it easy to build simple, reliable, and efficient software” from Google. But I’m also playing with GitHub, as the whole environment can be tightly integrated with it. And so, by extension, I’m playing with Git. More on all of those at a later, probably over on my Daemon Dancing In The Dark more generalized Linux blog.

Anyway, I’m working on a program I call Iceray. It is a command line application that will send MP3 files to an Icecast / Shoutcast server, like my Vehement Flame radio station. I wanted something I could run that would just feed the server without me handholding it.

So I went to work on packaging it up and finally got a PKGBUILD to work. I spent way too much time working on it, especially when it turned out the Go Packing Guidelines wiki page is pretty good. The sample PKGBUILDs should have worked better for me, but I got a few of my streams crossed which took me out of my game for a bit.

The first thing was actually in my Go program. I changed all the package declarations to be ‘package iceray’, not realizing that if it isn’t ‘package main’ it will build a library and not an application. And I also had the _gourl in my PKGBUILD wrong, in that I originally created it as a Git package, so I had the git: protocol in there.

And the page is still a little confusing, in that I needed to use a combination of the sample PKGBUILDs. While iceray in the end boils down to an application, I still found it necessary to use ‘go get‘ in the build() routine, because I don’t think ‘go build‘ pulls in the necessary remote libraries (I use go-libshout, gcfg, and go-id3). And the application ones use the path “$srcdir/$pkgname-$pkgdir”, but “$srcdir” worked just fine for me. So here’s the final PKGBUILD:

# Contributor: Jonathan Arnold <jdarnold@buddydog.org>

pkgname=iceray
pkgver=0.2
pkgrel=2
pkgdesc="Icecast/Shoutcast commandline client"
arch=('i686' 'x86_64')
url="https://github.com/jdarnold/iceray"
license=('MIT')
depends=('libshout')
makedepends=('go')
conflicts=('iceray')
provides=('iceray')
options=('!strip' '!emptydirs')
_gourl="github.com/jdarnold/iceray"
install=${pkgname}.install

build() {
    cd "$srcdir"
    GOPATH="$srcdir" go get -v -x ${_gourl}/...
}

package() {
    install -Dm755 "$srcdir/bin/$pkgname" "$pkgdir/usr/bin/$pkgname"
    install -Dm644 "$srcdir/src/$_gourl/iceray.gcfg.sample" "$pkgdir/usr/share/$pkgname/iceray.gcfg.sample"
    #install -D -m644 iceray.1 $(DESTDIR)$(MANPREFIX)/man1/iceray.1
}

So, in the end, the PKGBUILD was pretty simple, despite my flailing about. Hope this helps someone else!

AUR (en) – iceray.




Arch Linux Schwag site is back up (a comment on privacy)

Hey all,

http://schwag.archlinux.ca has been down for at least a week. I’m really sorry about that. I’ve been working on transferring to a new web host and they were having trouble getting all the Python dependencies I needed.

My old host was based in the United States, and while they provided exceptionally good service for the price, I have become uncomfortable with storing any data in the US. The websites are not private, of course. However I am also hosting e-mail on these domains. As a Canadian citizen, I am not covered by the NSA’s (possibly false) claim that they do not spy on US citizens, so I felt that I should store it in my own country.

I don’t do anything that is currently illegal, but I don’t know that these arrogant governments aren’t going to change their laws to restrict my freedoms. I may not be a person of interest today, but I am outspoken about freedom, privacy, and transparency. This is currently legal and acceptable, but that may not always be the case and I want to be proactive about keeping my private data out of the hands of those who would use it against me.




Jolla Mobile

I freaking love my mobile phone.

It was one year ago on 28 Aug 2012 that I bought my first smart phone, a used Nokia N900. I’ve read about different phones and I knew exactly what I wanted. The N900 came out in late 2009. It’s pretty old, but there’s no other phone like it.

It’s pretty much a mobile Linux computer with a pretty and simple phone interface slapped on top of it. Since Linux is all I use at home it’s a perfect match for me. It’s all the power and all the tools that I use on my desktop computer in a form that can fit in my pocket.

"Hello? Oh, hey! Yeah, I got my new Linux phone! It's JUST LIKE a computer?"

“Hello? Oh, hey! Yeah, I got my new Linux phone! It’s JUST LIKE a computer?”

It took me a while to really realize what made the N900 special. After all, Android is “Linux” too. The N900 is different in that it’s Linux like every other Linux operating system. It has GNU. It (mostly) follows things like the Linux Standard Base and Filesystem Hierarchy Standard. The package manager is “apt” from Debian. It’s Linux, just like I like it.

It’s also very different from the iPhone in that all N900 phones are unlocked, and anyone can become root and get full access to the device from the day they get it. That means I’m in control of my phone and everything on it. I can install anything I want, even older versions of software. I can develop software for it from any computer, including making software FOR the phone ON the phone. I wouldn’t want to, but the point is that I can.

The N900 had one successor, the Nokia N9, before Nokia moved all of their smart phones to the Windows Phone operating system. The people that worked on my phone left Nokia and started their own company called Jolla. And if you’re a rabid Linux nerd like me then you’ll be excited about the mobile phone they’re developing.

What makes the Jolla Mobile phone special? It’s open source Linux underneath with a shiny new easy-to-use user interface. It’s made by the same people that made the N900 and the N9. It runs Android applications (if you can’t live without some). And it comes out by the end of 2013. Yay!

…but sadly I probably won’t be able to get one when they do come out. (Boo!) It appears that all the pre-orders have been booked, and there might be some time before they are sold in the US.

It’s for the best I suppose. I need to save up money for it (unless Santa really loves me).

Until then, I’ve got pictures of the Jolla Mobile phone as my wallpaper on my N900. I installed all the ringtones and SMS tones from Jolla (they’re pretty cool). So until I get one, I can dream…

my-jolla




The importance of “One Obvious Way”

In the face of ambiguity, refuse the temptation to guess.
There should be one– and preferably only one –obvious way to do it.
Although that way may not be obvious at first unless you’re Dutch.

— Tim Peters, The Zen of Python
>>> import this

While Python has numerous advantages over other languages that make me reach for it first when starting a new project, I feel the two most important are its emphasis on readability and the community belief in Tim’s words above, “there should be one– and preferably only one –obvious way to do it.” While it’s rare to stumble upon a Python coder that doesn’t implicitly “get” this principle, I find that is frequently misinterpreted when I try to explain Python’s charm to outsiders.

This may be because I often misquote the principle as “the one best way” principle. Many languages have a “one best way” principle. Java, for example, insists that the best way to solve any problem is object oriented programming. Similarly, prolog insists that logic programming is the best way, and Haskell programmers suggests that functional is always the way to go.

Python is not like this at all. Python is pragmatically multi-paradigm. Python has a long history of stealing great ideas from other languages (and, given Guido’s genius, making them better). Python does the very opposite of insisting that there is a best way to do everything.

Orthogonally, I have often had my defense of this principle misinterpreted as having said, “Python is always the best solution to any problem.” This makes me sound arrogant and misinformed, and is patently false. I am keenly aware of Python’s shortcomings. My day job has me pushing Python’s limits on a regular basis, and often, it fails me.

No, Python is not always the best way. The “one obvious way” principle refers to something quite different. It’s not about the language or its libraries at all, it’s more about the programmer’s mindset. Given any specific problem, there are invariably multiple ways to solve it. A Python developer will start out by assuming that one of those assorted solutions is the best solution. The goal of both the Python language and the libraries developed by the community is to make that best solution the obvious one to implement.

This is a form of consistency. Just as pep-8 strongly encourages us to be consistent with all other Python developers in our coding style, the “one obvious way” principle encourages us to be consistent in our choices of algorithms, data structures, and design patterns. In an ideal world, this means that any two Python programmers, given the same problem, would write virtually the same solution. Further, the solution that they proposed would be the best possible solution for the given problem, and all other pythonistas would find it elegantly readable.

This never happens. However, I believe that Python programs are closer to this ideal than many (if not most) other languages. This cohesion means that Python code is often more maintainable, not just by the original developer, but by all members of the Python community.

Some might suggest that strict adherence to the “one obvious way” principle is narrow-minded and restricts experimentation. I heartily disagree. There is clearly not one best web framework in Python. There are numerous web engines available, some more suitable to specific purposes than other. And more are being written every day. I think this is because nobody strictly believes that the most obvious implementation has been found, yet. The poem says there should be one obvious way. It doesn’t say that there always is one. The Python community as a whole is working together to find these best solutions and make them into the obvious solution. A few problems are solved. If you need an iterator, Python’s got you covered. Others are still up in the air, as evidenced by the collection of web frameworks or packaging tools or xml parsers on pypi. Some have only been recently solved; the Requests library is a terrific example. And still more are completely unsolved. The most obvious way I’ve found to handle concurrency and parallelism in Python is: Use Golang. Kivy is a terrific library, but the obvious way to do Android development is still (currently) Java.

In summary, while the ideal of “obviousness” is far, far away, I believe that it is a worthy goal to pursue. I think the mindset attracts a certain kind of programmer, and that those programmers are all as delighted when they discover Python as I was almost a decade ago. I don’t think Python is the best tool for all jobs, nor do I think it is the most enjoyable tool for all programmers. However, for those coders who like a certain kind of obvious, readable, elegance, Python is the language of choice.




hmm

#!/bin/bash
#
# Author: jbsnake
#
# Start configure: Hopefully you’ve read this far
#
# outfile=/var/log/httpd/log_rotation
outfile=/var/tmp/log_backups/log_rotation.out
num_days=30
uncompress_days=7
bob=-1
# log_dir=/var/log/httpd
log_dir=/var/tmp/log_backups
log_dir_old=${log_dir}/old/
old_access=${log_dir_old}access/
old_error=${log_dir_old}error/

# End configure

cur_date=`date +%Y%m%d`
outfile=”${outfile}.${cur_date}.out”

function usage
{
echo “”
echo “# ‘$0 -0′ (dry run)”
if [[ $bob == 0 ]]; then echo -e “# ‘$0 -f’ (live run)\n# Note: Double-check other configs before using”; fi
echo “”
exit
}

function daysbetween
{
future=$1
past=$2
echo “$(( ( $(date -d “$future” +%s) – $(date -d “$past” +%s) ) /86400 ))”
}

if [[ $bob != 0 && "$1" != "-0" ]]
then
usage;
elif [[ "$1" == "-0" ]]
then
bob=1
fi

if [[ "$1" != "-f" && $bob == 0 ]]
then
usage;
fi

# making file structure
if [[ $bob == 0 ]]
then
mkdir -p ${log_dir_old}
mkdir -p ${old_access}
mkdir -p ${old_error}
fi

# start logging
echo “### Starting log rotation: `date` ###” >> ${outfile}
echo “# ” >> ${outfile}
if [[ $bob != 0 ]]; then echo -e “# *** Dry Run *** \n# To do live run, change bob to 0\n#” >> ${outfile}; fi
echo “# Details: ” >> ${outfile}
echo “# Number of days retention:” >> ${outfile}
echo “# ${num_days}: Total (compressed and uncompressed)” >> ${outfile}
echo “# ${uncompress_days}: Uncompressed” >> ${outfile}
echo “# Log Directory to be rotated: ${log_dir}” >> ${outfile}
echo “# Old Log Directory (where the logs go):” >> ${outfile}
echo “# ${old_access}: Old Access Log files.” >> ${outfile}
echo “# ${old_error}: Old Error Log files.” >> ${outfile}
echo “### ” >> ${outfile}

# enter log directory
cd ${log_dir}

# work on error_logs
for error_file in `ls error_log.*`
do
log_date=`echo ${error_file} | sed “s#error_log\.##g” | sed ‘s/\.//g’`
if [[ "${log_date:${#log_date}-2}" == "gz" ]]
then
iscompress_tag=true
log_date=${log_date:0:${#log_date}-2}
else
iscompress_tag=false
fi

age_file=`daysbetween $cur_date $log_date`

if [[ "$age_file" -lt ${uncompress_days} ]]
then
echo “# File: $error_file newer than ${uncompress_days} days, skipping” >> $outfile
else

if [[ "$age_file" -gt $num_days ]]
then
echo “# Removed: $log_date – $age_file days old} : $error_file ” >> $outfile
if [[ $bob == 0 ]]; then rm -f ${log_dir}/${error_file}; fi
else
echo “# Compressing and moving: $error_file to ${old_error}${error_file}.gz” >> $outfile
if [[ $iscompress_tag == false ]]
then
echo “# Compressing: $error_file” >> $outfile
if [[ $bob == 0 ]]; then gzip ${log_dir}/${error_file}; fi
error_file=${error_file}.gz
fi
echo “# Moving: $error_file” >> $outfile
if [[ $bob == 0 ]]; then mv ${log_dir}/${error_file} ${old_error}; fi
fi
fi
done

# reset age_file
age_file=”"

# work on access_logs
for access_file in `ls access_log.*`
do
log_date=`echo ${access_file} | sed “s#access_log\.##g” | sed ‘s/\.//g’`
if [[ "${log_date:${#log_date}-2}" == "gz" ]]
then
iscompress_tag=true
log_date=${log_date:0:${#log_date}-2}
else
iscompress_tag=false
fi

age_file=`daysbetween $cur_date $log_date`

if [[ "$age_file" -lt $uncompress_days ]]
then
echo “# File: $access_file newer than ${uncompress_days} days, skipping” >> $outfile
else
echo “# Compressing and moving: $access_file to ${old_access}${access_file}.gz” >> $outfile
if [[ $iscompress_tag == false ]]
then
echo “# Compressing: $access_file” >> $outfile
if [[ $bob == 0 ]]; then gzip ${log_dir}/${access_file}; fi
access_file=${access_file}.gz
fi
echo “# Moving: $access_file” >> $outfile
if [[ $bob == 0 ]]; then mv ${log_dir}/${access_file} ${old_access}; fi
fi
done

# get out of log dir
cd – >/dev/null

# get into old dir
# add a matching for loop for access logs (if you want to remove some)
if [[ `grep "### Log rotation completed:" $outfile` && `ls -R ${log_dir_old} | grep ".gz" | wc -l` -gt 0 ]]
then
cd ${old_error}

# reset age_file
age_file=”"

# work on old error logs
for error_old in `ls error_log.*.gz`
do
log_date=`echo ${error_old} | sed “s#error_log\.##g” | sed ‘s/\.//g’ | sed ‘s/gz//g’`
age_file=`daysbetween $cur_date $log_date`

if [[ "$age_file" -gt $num_days ]]
then
echo “# Removed: $log_date : $error_old : $age_file days old” >> $outfile
if [[ $bob == 0 ]]; then rm -f ${old_error}${error_old}; fi
else
echo “# Skipped: $log_date : $error_old : $age_file days old” >> $outfile
fi
done
cd – >/dev/null
fi
# add matching for loop here
echo “### Log rotation completed: `date` ###” >> ${outfile}
echo “” >> ${outfile}

echo “View status at: ${outfile}”




Creating an Application in Kivy: Part 9

Welcome to the final part of this series on creating a working Jabber client in Kivy. While there are numerous things we could do with our code to take it to Orkiv version 2, I think it’s currently in a lovely state that we can call “finished” for 1.0.

In this article, we’re going to talk about releasing and distributing Kivy to an Android mobile device. We’ll use a tool called buildozer that the Kivy developers have written. These commands will, in the future, extend to other operating systems.

We aren’t going to be writing a lot of code in this part (some adaptation to make it run on android is expected). If you’re more interested in deploying than coding, you can check out my state of the orkiv repository as follows:

git clone https://github.com/buchuki/orkiv.git
git checkout end_part_eight

Remember to create and activate a virtualenv populated with the appropriate dependencies, as discussed in part 1.

Table Of Contents

Here are links to all the articles in this tutorial series:

  1. Part 1: Introduction to Kivy
  2. Part 2: A basic KV Language interface
  3. Part 3: Handling events
  4. Part 4: Code and interface improvements
  5. Part 5: Rendering a buddy list
  6. Part 6: ListView Interaction
  7. Part 7: Receiving messages and interface fixes
  8. Part 8: Different views for different screens
  9. Part 9: Deploying your kivy application

Restructuring the directory

Back in Part 1, we chose to put all our code in __main__.py so that we could run it from a zip file or directly from the directory. In some ways, this is pretty awesome for making a package that can easily be distributed to other systems (assuming those systems have the dependencies installed). However, Kivy’s build systems presume that the file is called main.py.

I’m not overly happy about this. Kivy sometimes neglects Python best practices and reinvents tools that are widely used by the Python community. That said, it is possible for us to have the best of both worlds. We can move our code into orkiv/main.py and still have a __main__.py that imports it.

First, move the entire __main__.py file into main.py, using git:

    git mv orkiv/__main__.py orkiv/main.py

Next, edit the new main.py so that it can be imported from other modules without automatically running the app. There is a standard idiomatic way to do this in python. Replace the code that calls Orkiv().run() with the following:

(commit)

    def main():
        Orkiv().run()

    if __name__ == "__main__":
        main()

There are two things going on here. First, we moved the code that instantiates the application and starts it running in a function called main(). There is nothing exciting about this function. It does the same thing that used to happen as soon as the module was imported. However, nothing will happen now unless that function is called.

One way to regain the previous functionality of immediately running the app when we type python main.py would be to just call main() at the bottom of the module. However, that is not what we want. Instead, our goal is to make the application automatically run if the user runs python orkiv/main.py, but if they run import main from a different module or the interpreter, nothing is explicitly run.

That’s what the if __name__ conditional does. Every module that is ever imported has a magic __name__ attribute. This is almost always the name of the module file without the .py extension (eg: main.py has a __name__ of main). However, if the module is not an imported module, but is the script that was run from the command line, that __name__ attribute is set to __main__ instead of the name of the module. Thus, we can easily tell if we are an invoked script or an imported script by testing if __name__ is __main__. Indeed, we are taking advantage of the same type of magic when we name a module __main__.py. When we run python zipfile_or_directory it tries to load the __main__ module in that directory.

However, we do not currently have a __main__ module, since we just moved it. Let’s rectify that:

    from main import main
    main()

Save this as a new orkiv/__main__.py and test that running python orkiv from the parent directory still works.

You’ll find that the window runs, however, if you try to connect to a jabber server, you get an exception when it tries to display the buddy list. This is because our orkiv.kv file was explicitly importing from __main__. Fix the import at the top of the file so that it imports from main instead:

#:import la kivy.adapters.listadapter
#:import ok main

OrkivRoot:

While we’re editing code files, let’s also add a __version__ string to the top of our main.py that the Kivy build tools can introspect:

(commit)

    __version__ = 1.0

Deploying to Android with buildozer

Deploying to Android used to be a fairly painful process. Back in December, 2012, I described how to do it using a bulky Ubuntu virtual machine the Kivy devs had put together. Since then, they’ve designed a tool called buildozer that is supposed to do all the heavy lifting. Once again, I have some issues with this plan; it would be much better if Kivy integrated properly with setuptools, the de facto standard python packaging system than to design their own build system. However, buildozer is available and it works, so we’ll use it. The tool is still in alpha, so your mileage may vary. However, I’ve discovered that Kivy products in alpha format are a lot better than final releases from a lot of other projects, so you should be ok.

Let’s take it out for a spin. First activate your virtualenv and install the buildozer app into it:

    . venv/bin/activate
    pip install buildozer

Also make sure you have a java compiler, apache ant, and the android platform tools installed. This is OS specific; these are the commands I used in Arch Linux:

    sudo pacman -S jdk7-openjdk
    sudo pacman -S apache-ant
    yaourt android-sdk-platform-tools

Then run the init command to create a boilerplate buildozer.spec:

    buildozer init

Now edit the buildozer.spec to suit your needs. Most of the configuration is either straightforward (such as changing the application name to Orkiv) or you can keep the default values. One you might overlook is that source.dir should be set to orkiv the directory that contains main.py.

The requirements should include not only kivy and sleekxmpp but also dnspython, a library that sleekxmpp depends on. You can use the pip freeze command to get a list of packages currently installed in your virtualenv. Then make a conscious decision as to whether you need to include each one, remembering that many of those (cython, buildozer, etc) are required for development, but not Android deployment.

For testing, I didn’t feel like creating images for presplash and the application icon, so I just pointed those at icons/available.png. It’s probably prettier than anything I could devise myself, anyway!

If you want to see exactly what changes I made, have a look at the diff.

The next step is to actually build and deploy the app. This takes a while, as buildozer automatically downloads and installs a wide variety of tools on your behalf. I don’t recommend doing it over 3G!

First, Enable USB debugging on the phone and plug it into your development machine with a USB cable. Then tell buildozer to do all the things it needs to do to run a debug build on the phone:

buildozer android debug deploy run

I had a couple false starts before this worked. Mostly I got pertinent error messages that instructed me to install the dependencies I mentioned earlier. Then, to my surprise, the phone showed a Kivy “loading…” screen. Woohoo! It worked!

And then the app promptly crashed without any obvious error messages or debugging information.

Luckily, such information does exist. With the phone plugged into usb, run adb logcat from your development machine. Review the Python tracebacks and you’ll discover that it is having trouble finding the sound file in.wav:

I/python  (25368):    File "/home/dusty/code/orkiv/.buildozer/android/platform/python-for-android/build/python-install/lib/python2.7/site-packages/kivy/core/audio/__init__.py", line 135, in on_source
I/python  (25368):    File "/home/dusty/code/orkiv/.buildozer/android/platform/python-for-android/build/python-install/lib/python2.7/site-packages/kivy/core/audio/audio_pygame.py", line 84, in load
I/python  (25368):    File "/home/dusty/code/orkiv/.buildozer/android/platform/python-for-android/build/python-install/lib/python2.7/site-packages/android/mixer.py", line 202, in __init__
I/python  (25368):  IOError: [Errno 2] No such file or directory: '/data/data/ca.archlinux.orkiv/files/orkiv/sounds/in.wav'
I/python  (25368): Python for android ended.

The problem here is that the code specifies a relative path to the sound file as orkiv/sounds/in.wav. This worked when we were running python orkiv/ from the parent directory, but python for android is running the code as if it was inside the orkiv directory. We can reconcile this later, but for now, let’s focus on getting android working and just hard code the file location:

(commit)

    self.in_sound = SoundLoader.load("sounds/in.wav")

While we’re at it, we probably also need to remove “orkiv” from the location of the icon files in orkiv.kv:

    source: "icons/" + root.online_status + ".png"

Finally, running buildozer and adb logcat again indicates the error message hasn’t changed. This is because we aren’t explicitly including the .wav file with our distribution. Edit buildozer.spec to change that:

    # (list) Source files to include (let empty to include all the files)
    source.include_exts = py,png,jpg,kv,atlas,wav

Run the buildozer command once again and the app fire up and start running! Seeing your Python app running on an Android phone is a bit of a rush, isn’t it? I was able to log into a jabber account, but selecting a chat window goes into “narrow” mode because the phone’s screen has a much higher pixel density than my laptop. We’ll have to convert our size handler to display pixels somehow.

I was able to send a chat message, which was exciting, but when the phone received a response, it crashed. Hard. adb logcat showed a segmentation fault or something equally horrifying. I initially guessed that some concurrency issue was happening in sleekxmpp, but it turned out that the problem was in Kivy. I debugged this by putting print statements between each line in the handle_xmpp_message method and seeing which ones executed before it crashed. It turned out that Kivy is crashing in its attempt to play the .wav file on an incoming message. Maybe it can’t handle the file format of that particular audio file or maybe there’s something wrong with the media service. Hopefully the media service will be improved in future versions of Kivy. For now, let’s use the most tried and true method of bugfixing: pretend we never needed that feature! Comment out the line:

(commit)

    #self.in_sound.play()

and rerun buildozer android debug deploy run.

Now the chat application interacts more or less as expected, though there are definitely some Android related quirks. Let’s fix the display pixel issue in orkiv.kv:

(commit)

    <OrkivRoot>:
        mode: "narrow" if self.width < dp(600) else "wide"
        AccountDetailsForm:

All I did was wrap the 600 in a call to dp, which converts the 600 display pixels into real pixels so the comparison is accurate. Now when we run the app, it goes into “narrow” mode because the screen is correctly reporting as “not wide enough for wide mode”. However, if you start the app in landscape mode, it does allow side-by-side display. Perfect!

And that’s the app running on Android. Unfortunately, in all honesty, it’s essentially useless. As soon as the phone display goes to sleep, the jabber app closes which means the user is logged out. It might be possible to partially fix this using judicious use of Pause mode. However, since the phone has to shut down internet connectivity to preserve any kind of battery life, there’s probably a lot more involved than that.

On my phone, there also seems to be a weird interaction that the BuddyList buttons all show up in a green color instead of the colors specified in the Kivy language file and the args_converter.

Third, the app crashes whenever we switch orientations. This is probably a bug fixable in our code rather than in Kivy itself, but I don’t know what it is.

Also, touch events are erratic on the phone. Sometimes the scroll view won’t allow me to scroll when I first touch it, but immediately interprets a touch event on the ListItem. Sometimes touch events appear to be forgotten or ignored and I have to tap several times for a button to work. Sometimes trying to select a text box utterly fails. I think there must be some kind of interaction between Kivy’s machinery and my hardware here, but I’m not certain how to fix it.

Occasionally, when I first log in, the BuddyList refuses to act like a ScrollView and the first touch event is interpreted as opening a chat instead of scrolling the window. This is not a problem for subsequent touch events on the buddy list.

Finally, there are some issues with the onscreen keyboard. When I touch a text area, my keyboard (I’m using SwiftKey) pops up, and it works well enough. However, the swipe to delete behavior, which deletes an entire word in standard android apps, only deletes one letter here. More alarmingly, when I type into the password field, while the characters I typed are astrisk’d out in the field, they are still showing up in the SwiftKey autocomplete area. There seems to be some missing hinting between the OS and the app for password fields.

Before closing, I’d like to fix the problem where the app is no longer displaying icons on the laptop when I run python orkiv/. My solution for this is not neat, but it’s simple and it works. However, it doesn’t solve the problem if we put the files in a .zip. I’m not going to worry about this too much, since buildozer is expected, in the future, to be able to make packages for desktop operating systems. It’s probably better to stick with Kivy’s tool. So in the end, this __main__.py feature is probably not very useful and could be removed. (Removing features and useless code is one of the most important parts of programming.) However, for the sake of learning, let’s make it work for now! First we need to add a root_dir field to the Orkiv app. This variable can be accessed as app.root_dir in kv files and as Orkiv.get_running_app().root_dir in python files.

(commit)

    class Orkiv(App):
        def __init__(self, root_dir):
            super(Orkiv, self).__init__()
            self.root_dir = root_dir
            self.xmpp = None

Of course, we also have to change the main() function that invokes the app to pass a value in. However, we can have it default to the current behavior by making the root_dira keyword argument with a default value:

    def main(root_dir=""):
        Orkiv(root_dir).run()

Next, we can change the __main__.py to set this root_dir variable to orkiv/:

    from main import main
    main("orkiv/")

The idea is that whenever a file is accessed, it will be specified relative to Orkiv.root_dir. So if we ran python main.py from inside the orkiv/ directory (this is essentially what happens on android), the root_dir is empty, so the relative path is relative to the directory holding main.py. But when we run python orkiv/ from the parent directory, the root_dir is set to orkiv, so the icon files are now relative to the directory holding orkiv/.

Finally, we have to change the icons line in the kivy file to reference app.root_dir instead of hardcoding the path (a similar fix would be required for the sound file if we hadn’t commented it out):

    Image:
        source: app.root_dir + "icons/" + root.online_status + ".png"

And now, the app works if we run python orkiv/ or python main.py and also works correctly on Android.

There are a million things that could be done with this Jabber client, from persistent logs to managed credentials to IOS deployment. I encourage you to explore all these options and more. However, this is as far as I can guide you on this journey. Thus part 9 is the last part of this tutorial. I hope you’ve enjoyed the ride and have learned much along the way. Most importantly, I hope you’ve been inspired to start developing your own applications and user interfaces using Kivy.

Financial Feedback

Writing this tutorial has required more effort and consumed more time than I expected. I’ve put on average five hours per tutorial (with an intended timeline of one tutorial per week) into this work, for a total of around 50 hours for the whole project.

The writing itself and reader feedback has certainly been compensation enough. However, I’m a bit used up after this marathon series and I’m planning to take a couple months off before writing anything like this tutorial again in my free time.

That said, I have a couple of ideas for further tutorials in Kivy that would build on this one. I may even consider collecting them into a book. It would be ideal to see this funded on gittip, but I’d want to be making $20 / hour (far less than half my normal wage, so I’d still be “donating” much of my time) for four hours per week before I could take half a day off from my day job to work on such pursuits — without using up my free time.

A recent tweet by Gittip founder Chad Whitacre suggests that people hoping to be funded on gittip need to market themselves. I don’t want to do that. I worked as a freelancer for several years, and I have no interest in returning to that paradigm. If I have to spend time “selling” myself, it is time I’m not spending doing what I love. In my mind, if I have to advertise myself, then my work is not exceptional enough to market itself. So I leave it up to the community to choose whether to market my work. If it is, someone will start marketing me and I’ll see $80/week in my gittip account. Then I’ll give those 4 hours per week to tutorials like this or other open source contributions. If it’s not, then I’ll feel more freedom to spend my free time however I like. Either way, I’ll be doing something that I have a lot of motivation to do.

Regardless of any financial value, I really hope this tutorial has been worth the time you spent reading it and implementing the examples! Thank you for being an attentive audience, and I hope to see you back here next time, whenever that is!

Creating Apps In Kivy: The Book

My request for crowd funding above didn’t take off. I’m making just under $12 per week on Gittip, all of which I’m currently regifting. Instead of my open contribution dream being fulfilled, I opted to follow the traditional publishing paradigm and wrote a book Creating Apps In Kivy, which was published with O’Reilly. I’m really excited about this book; I think it’s the highest quality piece of work I have done. If you enjoyed this tutorial, I encourage you to purchase the book, both to support me and because I think you’ll love it!




Bad Guy Discrimination

On my flight back from an absolutely amazing Python Canada 2013, I watched “42″ (Jackie Robinson’s story) and about half of “Skyfall”. The former warmed my heart because the racial discrimination was so disturbing. You’re probably reading that twice and scratching your head, so let me explain: It’s heartwarming that we have made progress in fighting racism, and that things that were considered normal 70 years are truly disgusting today. While nauseating racism, sexism, homophobia, and other forms of othering still exist today, in Canada, at least, it is no longer considered normal. For example, last year, the NHL media was in uproar because of an alleged racial slur against PK Subban. Yes that slur should not have happened, but 60 years ago, it would have been encouraged.

Skyfall naturally reminded me of past decades of James Bond movies. Traditionally, the bad guy in a Bond film is a dude with an accent reflecting whatever country the American media (while Bond is British, the films are Hollywood) was hyperventilating about at the time. This “bad guy with an accent” portrayal is a form of xenophobia that Hollywood has been projecting for decades.

In Skyfall, as well as a majority of the very few other action movies I’ve seen in the past few years, the bad guy is not black like in the 70s, Russian like in the 80s, or middle Eastern like in the 90s. In movies such as Skyfall, Iron Man 2 and 3, all three Batman movies, and a super hero movie that I can’t remember the name of (citation required) the bad guy as a deranged local white man. Some of those insane white dudes have been phenomenal villains. Heath Ledger’s incredible portrayal of The Joker was particularly poignant.

Deranged local white men. These movies are silently teaching us, in the same way that movies of the 60s and 70s taught us that blacks are gangsters and women simper, that mentally ill people are dangerous, different, frightening. And you know what? We are! Even with the empathy I have developed from group therapy sessions and being hospitalized alongside seriously mentally ill individuals, I still get nervous when “crazy people” approach me on the street. I personally identified with The Joker, and my girlfriend at the time even commented that he reminded her of me.

The bad guy in a film normally has to be a little demented. I don’t want to live in a society where the desire to blow up entire cities of peace loving citizens is considered sane. I’m just here to remind people to encourage, rather than suspend, disbelief when watching all movies. Disbelieve the gender roles typified in Disney films. Disbelieve that people with accents are bad guys. Disbelieve that Asians can kill you with their pinky. Disbelieve that crazy men, men like me, are necessarily evil.




Get Blog Updates Delivered to your Inbox!
Enter your email address:

Delivered by FeedBurner

Add to Google Reader or Homepage

Subscribe in NewsGator Online

Add to My AOL

Add to netvibes

Add to Excite MIX