2018 in review: GDPR, and two native apps in three months

Two unrelated halves. The first was May 25th: every client, at once, in April, and a spring spent on consent, data minimisation and a command that scrubs personal data out of any database copy I pull to my machine. The second was the summer, when I wrote a native Android app in Java and a native iOS app in Swift, having never shipped either before.

The regulation itself is not the interesting part. What was interesting is that it forced me to answer a question I had been avoiding: which columns in this database are actually personal, and why is a copy of all of them sitting on my laptop?

Anonymising a production copy

If you work with a copy of a production database locally, everything privacy-sensitive in it has to go. I now ship an artisan command with the project that does exactly that. It always needs manual work, because only you can decide which columns count.

Take a plain User model. After pulling a copy you have every real customer, with firstname, lastname, email, password.

namespace App\Console\Commands;

use App\Models\User;
use Illuminate\Console\Command;
use Illuminate\Support\Facades\DB;

class CleanupPersonalData extends Command
{
    protected $signature = 'cleanup:gdpr {aggressive?} {force?}';

    protected $description = 'Cleanup personal data for local environments';

    public function handle()
    {
        if ('production' == config('app.env')) {
            abort(403, 'This command should only be run in local/testing environments!');
            exit;
        }

        // overwrite everything, or skip what was anonymised before?
        $force = 'true' == $this->argument('force');

        // prefix, so we can recognise anonymised rows later
        $anonymizeString = 'anonymized: ';

        $faker = \Faker\Factory::create('nl_NL');

        // a column to check for the prefix
        $mainColumn = 'lastname';

        // include soft-deleted rows, they hold personal data too
        $items = User::withTrashed();

        if (!$force) {
            $items->where(function ($query) use ($mainColumn, $anonymizeString) {
                $query->where($mainColumn, 'NOT LIKE', DB::raw($anonymizeString) . '%');
                $query->orWhereNull($mainColumn);
            });
        }

        $items = $items->get();
        $this->output->writeln(sprintf('%s users', $items->count()));

        // hash once, bcrypt is expensive
        $pass = bcrypt('123456');

        foreach ($items as $item) {
            if ($force || strpos($item->$mainColumn, $anonymizeString) === false) {
                $item->$mainColumn = $anonymizeString . $faker->lastName;
                $item->firstname = $faker->firstName;
                $item->email = $faker->safeEmail; // never mail a real address by accident
                $item->password = $pass;
                $item->save();
            }
        }
    }
}

Repeat the block per model, run php artisan cleanup:gdpr after every import, delete the original dump. Three details that matter more than the code:

• withTrashed(). Soft-deleted rows are still personal data. Skipping them is the mistake I made first.
• safeEmail. A test run that mails two thousand real customers is a very bad afternoon.
• The environment guard on line one. Not because you would ever run it on production deliberately.

Two apps, two languages I do not write

Between July and October I built the same product twice: Android in Java with Gradle, then iOS in Swift with CocoaPods. Not a webview wrapper around a website, which is what I would have quoted a year earlier. Actual native apps, twice, because in 2018 that was still what the two platforms required if you wanted it to feel right.

What I expected to be hard was the languages. It was not. Java and Swift are both perfectly reasonable, and after fifteen years of PHP the type systems were a relief more than an obstacle.

What was actually hard was everything around the code:

The build is the platform's, not yours. Gradle and CocoaPods are not npm. Signing keys, provisioning profiles, entitlements, target SDK versions. On the web I control the whole chain. On mobile I am a guest, and the host has opinions about certificates.

There is no deploying a fix. I had spent my whole career being able to correct a mistake in minutes. Suddenly there is a review queue between me and my users, and a copy of last week's bug sitting on every device that has not updated. It changes what "done" means, and it made me test in a way that web work never forced me to.

Doing it twice is doing it twice. Same screens, same logic, two codebases, and a fix in one is not a fix in the other. The 2008 lesson about copying a folder nine times, wearing a different hat.

I have not written a mobile app since. But three months of not being able to hotfix is the best argument for testing I have ever been given, and I brought it back to the web work with me.

The consent side

The cookie module was easier to build than to specify. The hard question was never technical: it was getting a client to say out loud which third-party scripts they actually needed. Most of the time the honest answer turned out to be "fewer than are currently on the site", and a couple of tags got deleted rather than consented.

In short

GDPR took the spring: a consent module and an artisan command that anonymises any database copy on import. The summer went to a native Android app in Java and a native iOS app in Swift.

The year in commits