Key programming books

A top 5 of programming or related topics books I've found most useful:

Pragmatic Programmer
The Mythical Man Month
Perl  Testing, a developers notebook
UML in 24 hours
The Design of Everyday Things


I'll blog details of each one later, in particular 'The Design of Everyday Things' which isn't an obvious 'programming' related book.

Dangerous design flaw in wordpress

Wordpress insists on storing fully qualified URLs all over the place. Especially in the database. This means that if you restore a backup of a wordpress installation onto another machine (a test/dev machine for example), you can unexpectedly end up being switched to the live site at any moment.

Add to this the fact that as previously complained about, the whole thing is stored in the database, so you are logging into a dev machine with the same user account as live, and the potential for catastrophe is huge.

fun with recursive chown and dotfiles

 rm -rf .*  

removes all hidden subdirectories and their contents, but generates an error and does nothing when it tries to remove '.' and '..'

I've always just accepted that behaviour without thinking. I now discover that it's the wrong thing to do. I discovered this by doing

 chown -R codehare:codehare .*  

This has a very unexpected effect. A very bad, very unexpected effect. It chowns everything that matches .* (i.e. hidden directories) in the current directory, then it starts in the parent directory and recurses through all subdirectories of that. This is not good news. Especially if you do it as root/sudo. Especially if you do it in a home directory. Especially if you use key based authentication.

Well that was a fun start to the week...

Wordpress - yet more 'everything done in live' horror.

I've just restored a copy of a wordpress DB from a colleagues dev machine. When I run it up locally none of the resources (CSS etc) are accessible because the base URL is wrong.

It turns out that all URLs are absolute, and written into the database. Tools apparently exist to traverse the database changing all these entries.

This really is appallingly noddy and unprofessional. It's no wonder web development as a discipline has such a dreadful reputation.

12 Factor

This is a good set of principles for neatly encapsulated services. In many ways it reads like the SOA special case of the Pragmatic Programmer. This is what SOA promised from the start

http://12factor.net/

When writing documentation

Tip for developers when writing documentation. Normal people (by which I mean non-developers) do not look at 5000 word blocks of solid text and think "yay! Lots of useful information!".

Diagrams in documentation don't have to convey any illuminating and helpful points. Sometimes they can just be a way of stopping your reader bursting into tears before the halfway point.

Everything that is wrong with web development is perpetuated by the market leading CMSes

As I've said before, I'm not a web developer. I work for a small software house though, and as a result I end up doing a fair amount of web development. It's an experience that always  leaves me wanting to wash. With bleach.

The last couple of years, focus on virtualization has started to move from the big, heavyweight server virtualization platforms like ESX towards virtualization of services or microservices. This is finally starting to realise the promise of SOA in smaller, non-enterprise environments. Docker can run containers on virtualized platforms like AWS to massively simplify deployment and management.

But when you start trying to use this in reality you run into the problem of the CMS. Customers want CMSes - they want to pay for us to set up their site, but have the option to do little tweaks and edits themselves. Web development on the smaller scale end, providing simple sites that are updated a couple of times a month for businesses of up to 20 staff or so; and ecommerce sites for marginal 'run from a bedroom' businesses, small shopfronts etc., is totally dominated by 2 CMSes - Wordpress and Joomla*. There's not a lot to choose between them: both backend into MySQL, both are written in PHP, both have a huge estate of plugins.

The worst problem with web development is that so much of it happens in production. We bemoan 'bedroom programmers' and the amateurish habits of a lot of web developers, but in reality this happens because of the CMS. The CMS stores page content in the database. When you are developing a site for a customer the 'product' you are producing is scattered across template files and database entries. Those database entries are stored on a database engine along with a load of other products. Yes, you can dig them out with database queries, but it's still a mess of jumbled string with no sense of encapsulation. This makes staged deployment impractical**.

This is why everything happens in production, because it's all but impossible to have proper staging and controlled release. All the amateurish shit that happens in web development logically flows from that one place.

Is there an answer? Probably. CMSes backending into MySQL are the source of the problem, especially at this level. Serving from files is faster, more reliable, more scale appropriate and makes it possible to use proper versioning tools. It's only inertia and the availability of plugins that keeps us using them. My favoured approach would be something like Grav running in a docker container, and since Grav has a shopping cart it should be straightforword to do.

Time to do some experimenting.

https://getgrav.org/

https://gravshoppingcart.com/

* Drupal is a bit bigger scale, but suffers from exactly the same problems.
** Yes yes, I know. Vagrant, Ansible, Puppet etc. Very powerful tools that, in this context, do a great job of making up for the design shortcoming that  MySQL backed CMSes don't encapsulate properly.

Android, invisible files, and folders that look like empty files

Ugh.

That took a long time, a lot of poking, swearing, reading stackoverflow, more poking, more swearing, and finally digging around in an open source file manager to find out how they'd done it.

So.

If you create a file on an Android device and attempt to view it with MTP/PTP, and it isn't there, you need to do this:

sendBroadcast(new Intent(Intent.ACTION_MEDIA_SCANNER_SCAN_FILE, Uri.fromFile(f)));

That tells the media scanner that it needs to know about this new file.

Cool. Great. Now, don't do this.

File f = new File(Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES), "MyStuff/");
f.mkdirs();
//THIS! DON'T DO THIS ON A DIRECTORY!
sendBroadcast(new Intent(Intent.ACTION_MEDIA_SCANNER_SCAN_FILE, Uri.fromFile(f)));

Or you'll end up with folders that you can't access from windows via MTP/PTP. They'll look like empty/unknown type files. Just do

File f = new File(Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES), "MyStuff/");
f.mkdirs();

and the folder will appear straight away as it should.

Reading android sqlite db - quick and hacky way

from platform-tools

adb shell run-as uk.co.myco cat databases/data.db > data.db

then sqlite3 the db file