iCloud Drive stuck uploading and downloading files in Mac OS X Mojave

Recently, my MacBook Pro, running Mac OS X Mojave, got into a state in which the Finder wasn’t uploading or download files to and from iCloud Drive. Since I couldn’t find any solution to this through Googling, I wanted to document how I solved the problem for others who may be experiencing the same.

I think, but can’t be sure, that the problem was related to allowing GitHub Desktop to store my repositories in the “Documents” folder, which is synchronized with iCloud Drive.

One of my repositories had literally tens of thousands of files, and even though I had deleted it long ago on my Mac, I discovered through troubleshooting that it was still present in iCloud Drive, when accessed via the web.

To solve the problem, after moving my GitHub folder outside of Documents, I then backed up all the files in Desktop and Documents on my Mac that I need, and disabled “iCloud Drive” in the iCloud area of the system preferences, and instructed the Mac to delete all the local files.

I then went into iCloud Drive via the website, and started deleting everything from there.

Step one was selecting my repo folder in Documents, and deleting it.

Step two was going into “Recently Deleted”, and then purging the folder from there.

But this turned out to be very confusing, because although the purging of the repo folder appears immediate in the browser UI, it’s actually a very long process that’s happening—in the browser! While that’s going on (with no UI feedback) other files and folders in “Recently Deleted” area appear grayed out. You can select them, but you can’t delete them.

After a while of frustration, an error message popped up saying certain files in my repo folder couldn’t be deleted, at which point the folder re-appeared in the “Recently Deleted” area. Ultimately, I had to go into the folder, and delete its contents in small batches. Once that whole folder was deleted, the grayed out files and folder then lit up, and could be deleted themselves.

So the fact that deletions are getting processed one by one in the browser, but there’s no UI to indicate that, can cause terrible confusion when trying to perform the kind of mass cleanup that I was doing.

In the end, when everything was finally deleted and purged, I re-enabled iCloud Drive on the Mac, and everything returned to normal.

Fake profiles on Upwork

When sourcing work at Upwork, we tend to look for staff who are located in Europe, so that we have the chance to work in our timezone and have face-to-face meetings. A big problem, however, is fake profiles—people who are obviously not from Europe, posting profiles pretending they are.

My recent encounter with “Thomas Wang”, presumably from “Ghent, Belgium” is a case in point:

How to selectively run Keyboard Maestro macros in a synchronized environment

A challenged I’ve always faced in running Keyboard Maestro on multiple Macs, is the maintenance of macros that are common to all, i.e. when updating a macro on one machine (say, changing the API keys of a service I’m accessing), I have to remember to go make the same updates on the others.

Keyboard Maestro provides a solution to this problem, by allowing you to synchronize your macros across multiple machines. Their implementation, however, and in contrast to say, Hazel’s folder-scoped implementation, is all or nothing—meaning that you can’t have macros on one machine that don’t exist on the others. And that can become a problem, especially with macros that are scheduled to run periodically.

Keyboard Maestro provides two approaches to address this problem:

The first is the ability to, for any given group (folder) of macros, to click, “Disable on this Mac”.

Unfortunately, there’s a number of shortcomings to this option. For example, anytime you add a new group of macros to a given machine, you have to remember to potentially go around disabling them on the others.

The second approach, and better in my opinion, is to condition the execution of any macro on the UUID (universally unique ID) of the machine on which the macro is running. Here’s an example of how this works.

The first step is to maintain a macro that determines the UUID of the current machine, and defines a list of named UUIDs for machines you’ll later be referencing. I run the following macro daily, and whenever I add a new machine, I’ll add its UUID to the list of named machines by temporarily running the disabled action that copies the current machine’s UUID to the clipboard.

With this in place, I can now condition the execution of other macros by machine. The following is an example of a macro that runs daily, and quits FaceTime on my MacBook Pro.

If I wanted this macro to run on two machines, I could add a second UUID check, and change the condition to “any”.

While this approach requires additional effort in creating your macros, it provides big benefits in being able to manage from a single machine, the conditioned execution of macros on all of your machines.

How to perform a currency lookup in a Numbers spreadsheet

Apple recently introduced in the Numbers spreadsheet the ability to pull live stock prices from the internet, making it now possible to track portfolio performance.

To access this feature, you use the STOCK function:

Since the feature pulls data from the Yahoo finance service, the symbols you should use for reference are those used at Yahoo. For most stocks that I’ve come across, the symbols are the same as those used at Google, but they do seem to vary slightly for non-US stocks and currencies.

To track the Euro/USD exchange rate, the symbol used at Yahoo is “EURUSD=X”, but using this symbol in the Numbers STOCK function returns an error. The solution, as I found in this discussion at Apple is to use the CURRENCY function:

Startup Survivorship Bias

One of the best, and most spot-on, comics from xkcd:

The case against identity politics

In episode 45 of his “Waking Up” podcast, I loved this piece by Sam Harris on identity politics, and wanted to capture it here for future reference:

Sam Harris:

As far as I can tell, becoming a part of a movement doesn’t help anybody think clearly, so I distrust identity politics of all kinds. I think we should talk about specific issues, whether it’s trade or guns or immigration or foreign interventions or abortion or anything else. And we should reason honestly about them.

And I’m not the first person who has noticed that it’s pretty strange that knowing a person’s position on any one of these issues, generally allows you to predict his position on any of the others. This shouldn’t happen. Some of these issues are totally unrelated. Why should a person’s attitude towards guns be predictive of his views on climate change? Or immigration? Or abortion?

And yet, it almost certainly is in our society. That’s a sign that people are joining tribes and movements. It’s not the sign of clear thinking.

If you’re reasoning honestly about facts, then the color of your skin is irrelevant. The religion of your parents is irrelevant. Whether you’re gay or straight, is irrelevant. Your identity is irrelevant. In fact, if you’re talking about reality, its character can’t be predicated on who you happen to be. That’s what it means to be talking about reality.

And this also applies to the reality of human experience, and human suffering. For example, if vaccines don’t cause autism, if that is just a fact—which is what the best science suggests at this point—well, then to argue against this view, you need data. Or a new analysis of existing data. You need an argument. And the nature of any argument is that its validity doesn’t depend on who you are. That’s why a good argument should be accepted by others, no matter who they are.

So in the case of vaccines causing autism, you don’t get to say, “As a parent of a child with autism, I believe X, Y and Z.” Whatever is true about the biological basis of autism, can’t depend on who you are. And who you are in this case, is probably adding a level of emotional engagement with the issue, which would be totally understandable, but would also be unlikely to lead you to think about it more clearly.

The facts are whatever they are. And it’s not an accident that being disinterested—not uninterested, but disinterested, meaning not being emotionally engaged—usually improves a person’s ability to reason about the facts.

When talking about violence in our society, again, the facts are whatever they are. How many people got shot? How many died? What was the color of their skin? Who shot them? What was the color of their skin? Getting a handle on these facts doesn’t require one to say, “As a black man, I know X, Y and Z.” The color of your skin, simply isn’t relevant information.

When talking about the data, that is, what is happening throughout a whole society, your life experience isn’t relevant information. And the fact that you think it might be, is a problem.

Now this isn’t to say that a person’s life experience is never relevant to a conversation. Of course it can be. And it can be used to establish certain kinds of facts. If someone says to you, “Catholics don’t believe in hell”, it’s perfectly valid to retort, “Actually my mom is a Catholic, and she believes in hell.” Of course there’s a larger question of what the Catholic doctrine actually is, but if a person is making a statement about a certain group of people, and you are a member of the group, you might very well be in a position to falsify his claim, on the basis of your experience.

But a person’s identity and life experience usually aren’t relevant, when talking about facts. And they’re usually invoked in ways that are clearly fallacious. And many people seem to be making a political religion out of ignoring this difference, so I urge you not to be one of those people, whether you’re on the left or the right.

2016 Sunway Sitges International Chess Festival

Over the past four years, we’ve been to chess tournaments around the world, and a common thread has been that tournaments outside Spain are generally better organized and executed than those in Spain. That experience dramatically changed, though, with our recent participation in the Sunway Chess Festival, in Sitges, Spain.

Beginning with the location, the 10-day event was held at the four-star Sunway beachside hotel in Sitges, one of the better-known resort towns near Barcelona.

https://www.flickr.com/photos/matthenderson/31745449841/

We were fortunate to have recently discovered, by chance, that there’s a high-speed “AVE” train from Malaga to Barcelona (for some reason, this route doesn’t appear on the RENFE websites), the existence of which actually made it possible for us to attend, as there weren’t planes that met our date requirements. Plus, I always find traveling by train somehow more relaxing than plane. In fact, with power outlets on each seat, I’m writing this blog article while returning home on the train right now!

Having experienced many tournament-hosting hotels, the Sunway was definitely one of the best. The rooms were large and comfortable, and ours even included a kitchen. The food in the dining room buffet was absolutely amazing. And the hotel staff were exceptionally—and I mean exceptionally—friendly and helpful.

https://www.flickr.com/photos/matthenderson/31052255123/

But what really set this event apart was the attention to detail. For example:

In both the A and B tournament groups, the organization printed personal information cards, with photos, for each participant, and which were placed on display each afternoon next to the playing boards.
Each evening, as soon as the following day’s pairings were announced, the organization would email each participant a copy of their opponent’s information card, supplemented with their opponent’s tournament results up to that day.
At the beautiful playing hall, the organizer provided free pens for game notation, as well as free water, and each player’s notation page was provided on a handy hard-surfaced clipboard.
17 boards were re-transmitted live on the internet.
In the main playing hall upstairs, which had spectacular views to the sea, the organization had set aside five special tables—“G1” through “G5”—where each day they would invite random participants from the downstairs tournament to play. (I’ve never seen such a thoughtful detail at a tournament before.)
The arbiters were extremely efficient and competent.
Supplemental activities were organized daily, including two FIDE-rated evening blitz tournaments, GM master classes and game analysis sessions, as well as paella cooking and cocktail preparation courses.
The tournament arbiters frequently updated the results on Chess-Results throughout the games, even during the blitz tournament! In fact, since they did that during the blitz tournament, it was more convenient to look for your next pairing online, than to cluster around the paper-printed pairings hung on the wall.
The delivery of the generous money awards after the closing ceremony was organized into several prize-related queues, for fast and efficient processing.
Up-to-date information about the day’s activities, including any changes to the nominal planning, were neatly printed in Spanish and English, and posted everywhere within the hotel.
The hotel provided passes for free use of the local bus system.
The hotel even provided free bicycles to those staying at the hotel, for use traveling into town.
On the last day of the event, each participant found a nice brown bag on their playing board, full of high-quality local delicacies as a good-bye present.

I want to emphasize that these kinds of details don’t happen accidentally. A thoughtful group of people took the time to identify each and every one of these details, and then plan the successful execution of each one. Just as it’s easy to overlook all the thinking and design that goes into your easy-to-use Apple product, it would be easy to overlook the care and effort that went into organizing the 2016 Sitges Chess Festival. So I’m here now to publicly recognize and express appreciation for the organizer’s efforts. Bravo!

https://www.flickr.com/photos/matthenderson/31052257373/

(One of the arbiters kicking off the event with a rendition of Sinatra’s, “I did my way!”)

As always, it was also good to see people from around the world that we only get to see from time to time. Lance got to see his American friend and soon-to-be Grand Master, Awonder Liang. I got to see some British friends I met a few years back at a tournament in Sevilla. We got to see GM Damian Lemos, who’s instructional videos we purchased back in the day.

In terms of chess level, this was one of the strongest events we’ve had the opportunity to participate in, comparable to the fabulous Tradewise tournament in Gibraltar. In particular, India sent an unusually large group of strong young players, i.e. kids in their teens with ELOs in the 2200 range who were playing at IM levels, and scoring draws against players with ELOs of 2500 and 2600!

Lance had a good tournament, drawing in the final round with GM Damian Lemos, but coming up short on achieving an IM norm due to one loss against a lower-rated player, and finishing the tournament with 5.5 of 9 possible points.

https://www.flickr.com/photos/matthenderson/31824196916/

(In Lance’s final game, he drew with GM Damian Lemos.)

Andrea played a very high level in all of her games, consolidating her 2000 ELO level, and finishing with 3 points.

https://www.flickr.com/photos/matthenderson/31745438371/

I played the B tournament, scoring 4 points, and finally increased my ELO above the 1600 mark. Personally, the highlight of my tournament was the last round, and getting to play a fellow American, Anthony Ciarlante, who’d traveled with a group from Shippensburg University in Pennsylvania to participate in the European event.

I played white against Anthony, and had the opportunity to play the explosive Evan’s Gambit. Shortly after the opening, I had a powerful attack on Anthony’s king, causing him to weaken his king-side pawns and cramp the development of his queenside pieces. When I missed a winning Qxh6 move—that Anthony later showed me in the analysis!—and ended up trading my attacking bishop for Anthony’s passive rook, he then returned fire with his own explosive counter-attack, and found a brilliant Qb4 move that appeared to win a piece. After thinking forever, and getting down to 15 minutes on the clock, I finally found Qxc7, move that both Anthony and I thought at the time was good, and actually led to me winning the game, but later in the analysis we both realized black had a good response!

Wow, what a game—and one in which it’s a pity anyone had to lose, as both Anthony and I agreed it was the funnest game of our tournaments. In case you’re interested, you can see the game over on ChessDrop.

All in all, we had a wonderful experience at the 2016 Sitges Chess Festival, and can’t wait to return in 2017! And with that, I’ll leave you with some photos from the event:

Flickr API error: Photoset not found

Email Verification

I’m the owner of a Gmail address that bears my name, in the form first.last@gmail.com.

Many others who share my name, have addresses that are slight variations of mine, e.g. first.last2@gmail.com or first3.last@gmail.com, or even first.p.last@gmail.com. You get the idea.

Often when these people signup at websites, they mistype their email address—and accidentally enter mine.

On account creation, modern websites send a verification email to the registered address, containing a link that the user must click before they can use the service. This verification-loop confirms that the person actually owns the email address they entered. You’ve probably experienced this yourself.

If I’ve directed you to this article, it’s likely because your company does not verify email addresses, such that I’m currently experiencing one or more of the following problems:

I’m receiving notifications, alerts, user-related communications (often containing personal data), and I’ve been unable to stop them because either:
- Your communications don’t have an unsubscribe link, or
- There is an unsubscribe link, but requires login to confirm
Your service doesn’t allow me to reset the account password simply by knowing the email address, i.e. it’s requiring me to provide some user-specific information I wouldn’t know.

In other words, I am stuck, have wasted time that I shouldn’t have wasted, and need your help.

But just as importantly, I need that you get the message to whoever in your company is responsible for the website, insisting that they need to add email address verification to the account creation process, to prevent this from happening in the future.

Thank you.

Follow-up thoughts about Goldmoney

On December 1, the Canadian company formerly known as BitGold, sent an email to their customers, announcing a change that fundamentally affects how their customers use and experience the service.

The essence of the message was that BitGold customers with holdings of more than 1,000 grams of gold could no longer use the service as place to continue accumulating gold savings. For those customers wishing to continue saving, they would need to create an account with GoldMoney, the UK company that BitGold acquired earlier in the year.

(Note that earlier this year, BitGold rebranded as “Goldmoney” to encompass both companies. In this article, however, I’m going to continue referring to them as BitGold to distinguish between them and the UK “GoldMoney” company. In actuality, BitGold now refer to themselves as “Goldmoney Personal & Business”, and refer to GoldMoney as “Goldmoney Wealth”.)

Customers forced to open GoldMoney accounts would experience a number of consequences and disadvantages with respect to what they were used to at BitGold, including:

Annual storage costs of 0.12% to 0.18%, when previously storage was free.
Gold purchase commissions of 2.5% on amounts under $10,000, when previously the commission was 0.5%. (GoldMoney doesn’t charge sell commissions, but your gold would have to appreciate by a factor of four before you’d break even with respect to BitGold.)
Loss of ability to purchase gold with a credit card.
Loss of the ability to make online vault-to-vault transfers.
A degraded online user experience.
You’ll likely have to go through KYC/AML procedures again.
~~Potentially, A 0.5% vault-to-vault fee to transfer gold in the BitGold network to the Toronto vault, which is the only vault available for transfer to GoldMoney.~~ (I’m relieved to report that they refunded my vault-to-vault fee. Yeah!)

Overall, this announcement was not terrible news. Compared to similar services, BitGold/GoldMoney is still competitive. And through the maintenance of accounts at both BitGold and GoldMoney, one can still maintain 1kg of gold for free at BitGold, and even avoid the high purchase commission by purchasing at BitGold, and transferring to GoldMoney.

But still, this announcement was definitely not good news for affected BitGold customers. And the company should have made that clear in its announcement. They should have said something like:

We’re announcing an important change that’s going to limit your continued use of BitGold, and require you to open an account with our sister company if you wish to continue accumulating gold. There will be some disadvantages, and the migration might be a little inconvenient. Here’s what you need to know…

But they didn’t. Instead, they said this:

We’ve worked to expand the capabilities of the Network and want to share the benefits of applying for a Goldmoney Wealth Holding, as storage fees will apply to Network account balances of or above 1,000 grams at the rate of 0.18% per annum as of January 1, 2017. Please note that you may still hold up to 1,000 grams in a Goldmoney Personal or Business account for free. As you presently hold a balance of 1,000 grams or more in your Network account, we invite you to apply for a Goldmoney Wealth Holding for any storage needs you may require over this amount.

This communication was ambiguous, confusing and played down how fundamental this change really is. It didn’t explain why the change had to happen, nor the impact it would have on affected customers.

Last week, I posted an article arguing that this was a botched communication on the part of BitGold, for not being clear and transparent. I’ve worked in the product industry for 25 years and have some experience in this area, and so I was surprised when BitGold CEO Roy Sebag responded¹ with what felt like a dismissal of my position:

In fact, I did approach them first. But at that point, none of the emails I’d sent to their support department since mid-November had been responded to, and the ones specifically about this announcement still haven’t!

After posting my blog article, and the ensuing Twitter conversation, I was contacted both publicly and privately by other BitGold/GoldMoney customers, expressing agreement with my sentiment, and disappointment in Roy’s response.

In fact, one customer went so far as to reduce his holdings as a result:

Since May of 2016, the Canadian company has lost 50% of its market capitalization, and in response to that, Roy posted an article on Medium two days ago, which to my eyes appears as dismissive of that situation as he was to my concerns, basically pointing to Graham’s famous remark that markets in the short term are voting machines. (Note that nobody ever refers to that remark when explaining the high value of their stock.)

I sense a pattern of dismissing problems, and that causes me concern for the future of this company. Of course, I could be wrong, and I don’t know how things look on the inside, but from an outsider’s perspective I think their public-facing position should quickly change to one of humble honesty and transparency, even if that means something like this:

We’re young. We’re growing. We’re trying to innovate in an old, creaky and heavily-regulated system. It might get a little messy as we figure things out, but we’re capable, and we’re onto something good—so bear with us, because in the long run you’ll be glad you did.

Here’s the full conversation: ↩

Decommissioning old email addresses with FastMail

The first business email address I used, [email protected], now almost twenty years old, is the source of 95% of the spam I receive. I no longer use this address, and would simply like to kill it, but every now the arrival of an important message reminds me that decommissioning it could result in missing something important.

Our company uses FastMail for email hosting, and the account has several domains aliased, including makalumedia.com. Chatting with FastMail support, I discovered that I could use their advanced “Sieve” support to effectively kill the address without risking to miss important emails.

Here’s how I did it:

In Mail.app, I created a smart folder that collected all mail addressed to [email protected] during the past 10 years (and which is not in my junk mail folder). This is the starting point of my list of “known senders” from whom I’ll continue to receive mails.
I exported this smart folder to a mailbox file on my Desktop
I then used the Mac app “eMail Extractor” to parse a list of all email addresses found in that file.
I then used BBEdit to clean up the list, leaving me with only a single copy of unique {domain}.{tld} entries.
I then created the following Sieve rule in my account at FastMail

if anyof(
    address :contains "To" "[email protected]",
    address :contains "To" "[email protected]"
  ) {
  if allof(
    not address :matches "From" "*domain1\.com",
    not address :matches "From" "*domain2\.com",
    not address :matches "From" "*domain3\.com"
  ) {
      reject "This email address no longer exists. 
      Please contact me through my blog at dafacto.com 
      to get my new email address.";
      stop;  
  }
}

This sieve triggers on any mail received on my old makalumedia.com addresses. It then checks if the sender is in my list of known senders (which in my real sieve is much longer than the above). If the sender is not in that list, it rejects the mail with a message to contact me through my blog to get my current contact information.

Since setting this up a few days ago, my spam has been reduced by probably 90%. The few that have gotten through were from senders on my known-senders list, and so I went and removed them from the list. So over time, my known-senders list will get cleaned of the few spammers who were present in the original list.

All in all, I’ve been super happy with Fastmail. Their service is well-designed, technically solid, and provides just enough geeky flexibility to do advanced stuff like the above. Well worth the money!

Support Authentication

When I signup for an online service, I like to use an email address that’s unique to that service, i.e. something like [email protected]. Email for my-special-domain.com is then configured to forward all incoming mail to my personal email address.

This allows me to do two things:

Know which services sell my address on to third-parties. (If I start getting spam on this domain, I can figure out where it came from.)
Kill any address for which incoming mail gets out of hand

This works fine, except for one problem, and a problem that shouldn’t exist:

Often when emailing [email protected], I’ll get a reply back indicating that—for “security” purposes—I must email support from the address associated with my account at the service.

What’s the problem with that? The problem is that the “from” address of my support enquiry provides absolutely no authentication or security at all, since email headers are dead-easy to forge.

Therefore, if a service wants to authenticate support conversations, there’s only one way to do it, and that is to provide an internal messaging system accessible only once a user authenticates into the service’s website. (Most financial institutions have this, since getting user authentication right is particularly important to them.)

I decided to post this to my blog, in order to have something I can conveniently point to in the future, when trying to convince these services that they’re misguided and causing unnecessary inconvenience to users who prefer to use throw-away addresses on their accounts.

A botched change of policy at Goldmoney

This is the story of how Goldmoney botched the communication of a fundamental change in policy, that ultimately may lead to me taking my business elsewhere.

Buying gold in the early days

As you know from reading my book, I’m a fan of the Harry Browne “Permanent Portfolio”, which promotes holding a quarter of your wealth in gold, and at least a part of which is held in physical gold.

Back in the day, there were two major online options for purchasing and storing physical gold—BullionVault and the original GoldMoney. Both were located in the United Kingdom, and each controlled over a billion dollars in gold for their respective customers.

While both services offered web-based access to their customers, they both felt old-fashioned and rigid. Neither company’s websites would win any kind of design award, although BullionVault’s was, in my opinion, thoughtfully useful. Both companies restricted incoming and outgoing funds to a single linked bank account. Account changes required phoning in. While GoldMoney allowed you to buy and sell directly with them, BullionVault implemented an “exchange” model, whereby you felt like a participant in a “marketplace”.

Both companies had similar revenue models—BullionVault charged commission on both buying and selling, while GoldMoney charged only on buys (though that commission was considerably higher). And both companies charged an annual fee to store your gold.

Enter BitGold, and free storage

Later, a third player emerged, located in Canada, which caught my attention—BitGold.

BitGold’s stated mission was to build technology that allowed one’s gold savings to be used as a currency. Compared to BullionVault and GoldMoney, BitGold felt innovative and modern. You could buy gold with your credit card, or even Bitcoin. You could link multiple bank accounts. You could transfer gold instantly between members. You could even spend gold in the form of a gold-backed MasterCard.

Having hired the talented and motivated designer, Mike Busby, the BitGold website looked and behaved like the kind of high-quality website my own company strives to build for its customers.

And from a cost point of view, BitGold was uniquely attractive in offering no storage fees:

Now, as you know from my book, long-term investors are particularly sensitive to annual fees, due to their destructive compounding effects over time, and so the absence of storage fees was a highly attractive selling point for BitGold.

In terms of revenue models, given that BitGold themselves would certainly have storage fees, it was speculated that their business model hinged on earning surplus revenue through their technology-driven value-add services.

BitGold jump starts their business through acquisitions

I didn’t initially open an account with BitGold, since they were still a new and unproven participant in the market, but that changed when the news emerged that BitGold had acquired GoldMoney, providing them with an considerable jump-start, and making them one of the world’s largest retail holders customer gold.

It was unclear at the time how existing GoldMoney accounts would be integrated into BitGold, but GoldMoney CEO James Turk’s statement certainly implied that the objective of the acquisition was integration:

We created GoldMoney with the vision of making gold accessible for savings and payments, a vision that BitGold is rapidly expanding in a new era of cloud computing and mobile technology. […] Users can expect a gold debit card, expanded payment options, as well as the many applications and features being developed by this innovative team.

One step towards this integration seemed to occur several months later when the company announced that BitGold was changing its name to “Goldmoney” (dropping the capital, “M”), and introducing three types of accounts.

My own account became a “Personal” Goldmoney account. A “Business” account was introduced for businesses. Finally, a “Wealth” account was introduced, that included storage fees, but benefitting from things like a special phone number to call for support, a dedicated “relationship manager”, the ability to hold money in several different currencies, and a couple of other features that I had no particular interest in.

It seemed the terms of my own, now “Personal”, account fortunately remained unaffected. Which was true for a while, anyway—bringing us to today, and to the point of this story.

A major change of policy at Goldmoney

Something has happened recently at Goldmoney, which has forced them to implement a major change in policy affecting Personal accounts:

While Personal accounts are still free of storage-costs, they are now limited to holding 1,000 grams of gold. Once you reach that limit, you are not allowed to purchase more, and are directed to open a separate Wealth account, which does apply storage fees of between 0.18% to 0.12% per year, and increases purchase commissions from 0.5% to 2.5%.
Existing Personal accounts with more than 1,000 grams of gold will not be forced to reduce their balance, but will have 0.18% storage charge applied to that portion of balance above 1,000 grams, and no further purchases can be made in the accounts.

This is a very big deal, for the following reasons:

Wealth accounts are held with a completely different company, i.e. the corporate entity behind the original GoldMoney service, located in the UK Channel Islands, and not the company behind the Personal accounts, located in Canada.

When you login to a Wealth account, you’re transported back in time to the original GoldMoney website, with only a visual update (currently in beta) to provide some consistency with the Goldmoney (nee BitGold) product.
Gold purchases are subject to a 2.5% commission, with no commission for sells. For comparison, BullionVault (and Goldmoney Personal accounts) charge 0.5% on buys and sells, and so your Goldmoney Wealth account purchases would need to appreciate by a factor of four before you’d come out better there. Storage fees between the two are nearly identical.
Unless your Personal holdings are stored in the Toronto vault, it’s going to cost you a 0.5% vault-to-vault transfer charge to move funds from Personal to Wealth.

This seems to change the basic Goldmoney service proposition to something sort of analogous to “checking” and “savings” accounts with a bank—i.e. you’re to hold your savings in a Wealth account, and any funds you want to use for low-fee transacting should be held in a balance-limited Personal account.

But while all this represents a fundamental, and arguably unfortunate, change, it’s not what I’m here to complain about. What I’m here to complain about is the way in which Goldmoney has communicated this change, which relates to the issue of trust.

Spinning a story

It would not be surprising to learn that BitGold/Goldmoney couldn’t manage to earn enough through their technology offerings to cover their own gold storage costs. Clearly, a company has to be profitable to be sustainable in the long-term and so even though it was good while it lasted, I can understand the need to apply storage costs.

What I dearly wish, though, is that Goldmoney had simply communicated the changes in a straightforward plain-English way, respectful of their customers ability to understand the reality of the situation.

Instead, this is the email I received today, which appears to spin the story as if the whole change is somehow “a great thing!”, and in the process, introduces confusion, and leaves many questions unanswered:

After leading with a positive-trending graphical chart and congratulatory message, the communication of the fundamental change in policy is delivered in this perfect example of corporate-speak:

We’ve worked to expand the capabilities of the Network and want to share the benefits of applying for a Goldmoney Wealth Holding, as storage fees will apply to Network account balances above 1,000 grams at a rate of 0.18% per year as of January 1.

Whatever this mess of a sentence is saying, here’s what it does not communicate:

You can NOT continue with your Personal account—If you’re currently a Personal account holder with over 1,000 grams, you must open a Wealth account in order to continue saving. There is no option to continue with your Personal account.
The whole Goldmoney experience changes—If you decide to continue saving with a Wealth account, the whole experience will change. The company changes. The website changes. The funding methods will change (no more purchasing with credit cards). The costs will change.
The transition may cost you—Depending where your Personal funds are vaulted, it may end up costing you 0.5% to move any excess balance from your Personal account to a new Wealth account, in order to get the storage fees down from 0.18% to 0.12%.
Why this all had to happen—Most importantly, the email doesn’t explain why we’re being subjected to this change. Reading it, you’d simply get the impression that, despite things going really, really well at Goldmoney, they’ve just decided to start charging Personal account holders with storage fees, above the arbitrary balance of 1,000 grams.

To understand any of the above (except for the justification), you have to login to your Personal account, and try to make sense of all the error messages and alerts. Overall, this was a botched communication.

Why this matters

Here’s a quote from 37 Signal’s book, “Getting Real”, from the chapter entitled “Publicize your screwups”:

If something goes wrong, tell people. […] Be as open, honest, and transparent as possible. Don’t keep secrets or hide behind spin. An informed customer is your best customer. Plus, you’ll realize that most of your screwups aren’t even that bad in the minds of your customers. Customers are usually happy to give you a little bit of breathing room as long as they know you’re being honest with them.

Although the change at Goldmoney isn’t necessarily the result of a screwup, it’s a major event that completely changes the product experience for savers, involves an inconvenient transition, and potentially involves costs.

Goldmoney’s communication should have clearly addressed these things. When you break your promises to someone, you better explain why. But it didn’t. Instead, it spun the story as a positive event, downplayed how fundamental the change is, and left many questions unanswered. And the result of that approach is twofold:

First, it leaves me to figure out all the consequences of the changes myself. It was only when I logged into the website that I realized I could no longer use the Personal account. It was only when I started to create a Wealth account that I realized it’s a completely different company, with a different website that lacks many of the features of the Personal account. And it was only when I started the migration process that I realized costs will be involved.
Second, it introduces distrust, and damages my confidence in this company. Is this how I can expect to be treated again in the future if/when they’re required to make major changes that affects me?

The whole situation has left me with such a bad taste in my mouth that I’m quite likely to take my business elsewhere. I hope this story finds its way to those in charge at Goldmoney, and that the folks working in PR there learn the lesson that it’s OK to tell things as they are.

Update 1 of 2: A week after posting this article, I posted some follow-up thoughts including a Twitter conversation I had with Goldmoney’s founders.

Update 2 of 2: To clarify what I would have liked to have received, I’ve drafted an alternative version of the Goldmoney email.

As a Goldmoney customer with a Personal account balance
above 1,000 grams, we need to communicate some changes that
affect you.
 
As background, we now offer three types of
accounts—Personal, Business and Wealth. In broad strokes,
our intent is that Personal accounts are not used for
long-term savings, but rather for transactions, i.e.
spending gold as currency. For savers and long-term gold
investors, our intent is for you to use our Wealth accounts.
 
To get from where we are today to that vision described
above, some changes have to take place, and so here’s what
you need to know and how you’ll be affected:
 
- Previously, there was no limit on the amount of gold you
could hold in a Personal account. We now limit Personal
accounts to 1,000 grams.Since your account has more than
1,000 grams, we’re going to start charging a 0.18% annual
storage fee on that portion of your balance above 1,000
grams.
 
- Although you can leave your balance as is, we will not
allow you to purchase more gold in your Personal account.
To purchase additional gold, you will need to create a
Wealth account.
 
- A Wealth account is held with an affiliated, but
different company, so you’ll need to go through KYC/AML
procedures again.
 
- You can link your accounts for the purpose of moving gold
from Personal to Wealth. But only
Toronto(Personal)-to-Toronto(Wealth) vault transfers are
permitted, so if your Personal gold is held in another
vault, you’ll first have to transfer it to Toronto, which
will result in a 0.5% vault-to-vault transfer fee.
 
- Wealth accounts have storage fees of between 0.12% and
0.18%, and purchase fees of 2.5% for purchases below
$10,000. They do not have sell fees.
 
- You can avoid those 2.5% buy fees, however, with a little
trick—i.e. you can transfer enough gold from Personal to
Wealth to free up some balance in Personal. Then, you can do
your purchasing in Personal, and transfer the proceeds to
Wealth at no cost.
 
We know this is a bit inconvenient, and possibility
confusing. We needed to make this change, however, to align
our account types with our different lines of business
strategy. If you have any questions about these changes,
don’t hesitate to contact us.

As a Goldmoney customer with a Personal account balance above 1,000 grams, we need to communicate some changes that affect you. As background, we now offer three types of accounts—Personal, Business and Wealth. In broad strokes, our intent is that Personal accounts are not used for long-term savings, but rather for transactions, i.e. spending gold as currency. For savers and long-term gold investors, our intent is for you to use our Wealth accounts. To get from where we are today to that vision described above, some changes have to take place, and so here’s what you need to know and how you’ll be affected: - Previously, there was no limit on the amount of gold you could hold in a Personal account. We now limit Personal accounts to 1,000 grams.Since your account has more than 1,000 grams, we’re going to start charging a 0.18% annual storage fee on that portion of your balance above 1,000 grams. - Although you can leave your balance as is, we will not allow you to purchase more gold in your Personal account. To purchase additional gold, you will need to create a Wealth account. - A Wealth account is held with an affiliated, but different company, so you’ll need to go through KYC/AML procedures again. - You can link your accounts for the purpose of moving gold from Personal to Wealth. But only Toronto(Personal)-to-Toronto(Wealth) vault transfers are permitted, so if your Personal gold is held in another vault, you’ll first have to transfer it to Toronto, which will result in a 0.5% vault-to-vault transfer fee. - Wealth accounts have storage fees of between 0.12% and 0.18%, and purchase fees of 2.5% for purchases below $10,000. They do not have sell fees. - You can avoid those 2.5% buy fees, however, with a little trick—i.e. you can transfer enough gold from Personal to Wealth to free up some balance in Personal. Then, you can do your purchasing in Personal, and transfer the proceeds to Wealth at no cost. We know this is a bit inconvenient, and possibility confusing. We needed to make this change, however, to align our account types with our different lines of business strategy. If you have any questions about these changes, don’t hesitate to contact us.

Disappointing interaction design at Apple

Long-time Apple customers became accustomed over the years to thoughtful and delightful interaction design. As Apple has grown—and perhaps as Steve Jobs has passed, and Jony Ive’s involvement seems to be sunsetting—cracks have begun to appear.

Here’s two examples I ran into just this morning.

Enabling Do-not-disturb in Notification Center

Trying to enable “Do-not-disturb” in Notification center, I ran into to problems:

First, it’s not clear to me which of these tabs are active.
Second, it took me a while to figure out that the Do-not-disturb control is only exposed when scrolling down in the notification. There’s no UI cues at all to help with discoverability here.

Assigning a photo to a contact

In the Contacts.app, when trying to assign a photo to person, you’d think the picker would default to the contents of your People album, and provide a usable UI for finding and selecting someone (with sensible fallbacks in the case you never configured any Faces.) Instead, we’re dropped into the root level of the photos hierarchy, and by the time we navigate to the People album, are left with a list that’s only capable of showing the first few letters of first names.

Isn’t there someone at Apple whose job is just looking around for these kinds of details, that are such an integral part of the brand we’ve come to perceive?

Email, please

Because every WhatsApp “informational” group eventually devolves into a water cooler chat, my app badge currently shows 1,457 unread messages. Because I rarely open Facebook, its app badge shows 312 unseen notifications. Because I participate in seven Slacks, with several “channels” in each, there are currently 117 unread messages in there.

Add in iMessage, Skype, Basecamp, Google Hangouts, Telegram, Signal & Twitter DMs, and we have a clear situation of contact-point overload.

If it’s important that I see what you have to say, there’s only one reliable channel—email. If you contact me through any of the others, let me apologize in advance—because I probably won’t see it.

The difference between developers and product designers

Our company is bidding on the re-development of an existing product that has outgrown the technical framework on which it was originally built. The customer has received a handful of offers, and the range of costs and technologies found in those proposals is causing him considerable uncertainty in his choice.

In response to that uncertainty, we’re nudging him to look beyond whether to use Ruby on Rails (our choice), Meteor or Laravel since, at the end of the day, the success or failure of his business will not hinge on technology. Instead, we’re encouraging him to consider the difference between a developer and a product designer, and focus on the critical question of who is capable of creating a product that will ultimately prove successful to his business.

To illustrate, let’s consider what happens when you signup for an account in their existing platform. Upon first login, you see something like this:

The original specifications for this product probably contained something along the lines of, “The system will have an accounts screen that lists all colleagues associated with the organization.” The developer then went about the task of satisfying the requirements, thinking:

When the account screen is accessed, I’ll query the database for all colleagues. And to account for the case there are no colleagues, I’ll show the message, ‘No colleagues found’.

Most developers focus on requirements and technology—i.e. the database query, the message to show if the query returns nothing, etc.—and fail to reflect deeply on the actual use of the product they’re building. In this case, the developer didn’t consider the one instance—and a critical one in terms of product success—of an empty database query that every single user will experience—Their very first engagement with this screen as a new user.

As a new user in this system, I’m left disoriented and confused:

Where am I, and what am I supposed to do?
The “No colleagues found” text seems like an error message. One minute in, and I’ve already done something wrong?
“Show blocked colleagues?” What is a blocked colleague? If I click that, the only thing that happens is that the text changes to “Hide blocked colleague”.

Had I created this account as a potential new customer wanting to “kick-the-tires,” there’s a good chance that I’d leave and not return, since experiencing friction in my very first interaction with the product is probably a good indication of what’s to come should I stay.

A good product designer is continually putting himself or herself in the shoes of the user, taking into account their context, their mindset, their knowledge and expectations, and looking to resolve any aspects of interaction with the product which potentially introduces friction.

In this example, a good product designer would identify the need for a “blank slate” version of the account screen, that’s welcoming and orientating for first-time users. Perhaps something like:

And therein lies the enormous difference in value between the average developer, and the very few who are good product designers. The former creates collections of features that “satisfy requirements”, while the latter creates coherent, effective and ultimately successful products.

Why keeping it simple would be a better choice for TransferWise

Overview

In the world of web application development, we sometimes face technical decisions whose trade-offs extend beyond the technical. Those non-technical trade-offs can be subtle, and perhaps difficult to identify, yet critical to the business.

In this article, I want to highlight as an example my experience with the TransferWise payment system, in which technical decisions ultimately work contrary the core of the product.

Background

In the early days of web applications, browsers like Firefox and Safari could only render web pages whose contents were structured in HTML and possibly styled with CSS. Any “logic” that formed part of the application had to be executed on the server.

Whenever you clicked a link on a screen, you’d experience a page refresh as the browser sent the request data back to the server, waited for the server to perform any necessary checks and calculations related to the request, and then your browser would display the HTML/CSS that was returned by the server.

So in those days, your browser only displayed things; any “thinking” happened on the server.

Time progressed, and browsers gained the ability to execute JavaScript software, thereby opening the door to implementing “logic” that gets executed within the browser client itself.

One of the most common first uses was in signup forms, as the browser could check that your entered-twice passwords matched, without requiring a page refresh and request to the server application.

Things got even more sophisticated when the browser could make a server request that’s transparent to the user. You’ve probably seen that when entering your username in a signup form, seeing a small spinner appear to the right, followed by a green checkbox informing you that, “Yeah, that username is still available!”

As “front-end” technologies continued to evolve over the years, we’ve gotten to the point where entire web applications are implemented in JavaScript, and run within the browser.

So, today, a fundamental decision to be taken by a developer when he or she implements a web application is:

Should I implement this logic on the server, or in the client?

The argument I want to make in this article, is that often this decision should be taken by the organization, and not simply left to designers and developers.

Context is everything

The benefit of using client-side logic is generally a smooth and seamless user experience, since the user doesn’t have to wait for page refreshes. The trade-off, however, is the risk of bugs in the user interface, since the JavaScript and rendering engines between different browser (and even different versions of the same browser!) can vary considerably.

There are some application contexts in which the risk of interface bugs is compensated by the value of a seamless and interactive user interface:

For example, if you’re developing a fast-paced interactive game, it could well make sense, in the interest of a smooth user experience, to implement the entire product as a client-side application.
Or let’s say you’re implementing product that’s likely to be used by your customers several times daily. In that case, saving a few screen refreshes might materially improve the experience when compounded daily over the period of an entire year.

At the same time, there are some application contexts in which a seamless user interface does not compensate the risk of exposing the user to interface bugs. And, here, I want to highlight an example of a company that has absolutely taken the wrong decision in this regard.

Disruption of an industry

In the past, it was terribly expensive for me to pay European contractors from my American company. First, the transfer itself would cost about $30. But then, I’d lose over 3% with respect to the market rate when the bank would convert my USD source funds to the destination currency of Euro.

TransferWise completely disrupted the market of moving and transferring money internationally, charging a fraction of what banks charge. They do this by taking advantage of volume to avoid even having to make transfers, i.e. if Customer A in the US transfers $100 to someone in Europe, and Customer B in Europe transfers the equivalent of $100 to the US, TransferWise can make the two transfers happen simply through off-setting accounting entries, using Customer A’s money to pay Customer B’s recipient, and vice versa.

What is the TransferWise product?

So what is the TransferWise “product”? If you ask me, it’s the saving of tremendous time and costs when making an international transfer.

And here’s where TransferWise have really messed up. They additionally view their “product” as the experience of making a transfer, and from a front-end technology perspective, they have decided that a slick user interface compensates the risks of exposing their user to bugs associated with the heavy use of front-end technologies.

To be specific: The process of making a transfer with TransferWise involves five steps:

You specify the source and destination currencies, and the amount to be transferred.
You choose who is sending the money (in case you happen to have both a personal and business profile on record).
You choose a recipient from a list of existing contacts, or create a new one.
You choose how you’ll get the money to TransferWise, e.g. through an ACH or wire transfer from your bank.
You review the transaction, and confirm if everything looks good.

Looks simple enough, but there’s quite some logic that has to happen:

You have to compute the amount of the conversion from the source to destination currency, based on the current rate.
You have to alert the user in case that rate expires during the process of setting up the transaction (i.e. if they take too long.)
You potentially have to walk the user through the “new contact” workflow.
You have to flag the user if the chosen receipt doesn’t have address details on file.
You have to walk the user through the “link new bank” workflow in the case they want to do an ACH transfer with a bank that wasn’t previously associated to their account.
You have to exclude the ACH option if the daily limit has already been exceeded.

So the process of initiating a transfer can get surprisingly complicated.

TransferWise’s flawed decision

Well, TransferWise decided to implement the entire workflow in one single web page, in which each step in the process is contained within its own component, that opens and closes accordion style.

The consequence of this approach, as opposed to pushing all the logic and checks to the server in page refreshes, is that during the entirety of my use of TransferWise, over the past few years, I have ran into user interface bugs probably more than 50% of the time.

And sometimes we’re talking about showstoppers—i.e. bugs that, in the name of a slick user experience, actually prevent me from making a transfer!

For example, the day that the confirmation button simply wouldn’t activate. Or the day when state wasn’t tracked across components and the confirmation button didn’t provide on-click feedback, such that multiple clicks of the confirm button suddenly skipped you multiple steps ahead in the process, leading to all sorts of chaos.

Or, what happened to me today…

I use TransferWise once a month to pay my European contractors. Since the only thing that changes each month is the amount I pay to each, I could really use “payment templates”. But since those don’t exist in TransferWise, the next best thing is to click “repeat payment” on some previous transfer, and then change the amount.

But it would seem that this isn’t the intended purpose of “repeat payment”, since clicking the option takes you directly to the confirmation component of the transaction screen. You can click back into Step 1, in order to change the amount, but I suspect my particular use of this feature is what caused me to see this, when finally returning to the confirmation component:

Try what again? Going from Step 3 to Step 4? Everything looks fine. What’s the problem?!

Neither refreshing the page, nor clicking “Confirm” removes the error message or allows me to proceed As with most of these UI errors at TransferWise, it would appear that I’ve reached a dead end.

But, in this case, guess what? When I return to my accounts page, I see that the transaction was successfully processed. So the error I was shown—i.e. the one that blocked the whole process—is itself erroneous!

Again, it’s all about context

So let’s backup and think about this.

Once per month I need to make some transfers.
I use TransferWise for this because they are fast, and save me a lot of money.
I do not use TransferWise because their transfer creation workflow is better than my bank’s. I don’t give a shit about that. If this were something I did 12 times per day, then maybe; but this is something I do 12 times per year.

Of course, it’s not impossible to have a reliable application that’s front-end heavy. It’s just that it’s much, much easier to have a reliable application that’s not. And in the case of TransferWise, a slick front-end doesn’t contribute to the core value proposition of the product, and my own experience demonstrates that there’s definitely inadequate value compensation in unnecessarily taking the risk.

Conclusion:

The very very last thing I want to experience in this context, are bugs that prevent me from making my transfer.

For the past year or so, each time I’ve experienced a UI bug, and have reported it to TransferWise, I’ve also take nthe opportunity to encourage them to reduce their dependence on front-end technologies, and give priority to making the process of creating a transfer as reliable as possible. But each time, missing the forest for the trees, their team have instead focused on trying to track down the particular bug I’m reporting (Have you tried that in Chrome?)

And so my hope is that through publishing this article, the larger issue might cross the radar of someone in TransferWise management, who’s in a position of considering the broader product goals.

2016 European Youth Chess Championship in Prague

The 2016 European Youth Chess Championship took place in beautiful Prague, capital city of the Czech Republic, about eight kilometers outside the city at the historic Top Hotel. Lance and Andrea participated as part of the 30+ kids playing for the Spanish national team.

https://www.flickr.com/photos/matthenderson/29337230525/

We’ve now had the opportunity to participate in two world championships and three european championships, and found the Top Hotel location to be among our favorites. It was close enough to the city that access via metro was a short 10-minute trip, but far enough away to feel secure in having the kids running around.

The hotel, whose interior was far more classic and majestic than its nondescript exterior, accommodated the 1500 or so participants surprisingly well. For example, the dining rooms at these events usually feel like chaotic stampedes, but that wasn’t the case at all in Prague.

https://www.flickr.com/photos/matthenderson/29337247605/

Each day’s round of chess play began at 3:00 pm. The time control was 90 minutes, plus an additional 30 minutes after move 40, resulting in games lasting up to four or five hours. Here Lance is playing at table 1 against top seed, Andrey Esipenko from Russia (the game ended in a draw).

https://www.flickr.com/photos/matthenderson/29337241865/

Trying to get in some exercise, my daily routine included, just after the round started, walking the 7.5 kilometers from the hotel to the Prague city center. The walk—which I could have never discovered without the help of Apple Maps—took about an hour and fifteen minutes, given that I’d stop every 15 minutes or so to check the games at the Chess24 retransmission site.

(As a side note, I took great advantage of the new laws requiring free data roaming throughout Europe. Combining that with Vodafone’s summer double-data promotion, I had nearly 20 GB of 4G data available for the trip. No more worries about crappy hotel wifi!)

Upon arriving to the outskirts of the city, I’d usually stop at La Bohème Café for a delicious iced raspberry tea or cappuccino, and catch my breath before heading further on into the city center.

https://www.flickr.com/photos/matthenderson/29337240265/in/album-72157669978997814/

Prague turned out to be one of the most beautiful and fascinating cities I’ve visited. Apparently, it was saved from the bombings of world war II, preserving its centuries-old architecture. The streets were lively, and bustling with summer visitors from around the world.

https://www.flickr.com/photos/matthenderson/28714094164/

https://www.flickr.com/photos/matthenderson/29049780320/

https://www.flickr.com/photos/matthenderson/29337291105/

After nine rounds of play, Lance finished the tournament 13th overall in the U14 boys category, tied in points (6.5) with the 6th place player Shant Sargsyan, but ending up in a lower position due to the tie-break calculations. He was happy with his play, although admitted to feeling a little tried after having played five tournaments and more than 50 rated games during the months of July and August!

The winner of U14 boys category, with 7.5 points, was Salvador Guerra, who, as it happens, is a friend of Lance’s from the same chess club here in Marbella! Just a few weeks earlier, after Lance won the U14 Spanish National Championship, Salvador won the U16 championship! It’s amazing that two kids playing at this level happen to come not only from the same region of the country, but also from the very same town!

Andrea finished her tournament with five points out of nine, finishing in 40th place overall.

Our flight back to Spain left a bit later than most everyone else, and it was a bit sad to experience the empty hotel on the morning following the awards ceremony and saying goodbye to chess-friends we see only once a year. We had a great time at this event, and the location couldn’t have been better. Lance returned home with a burning desire to start his next phase of chess study, while Andrea plans to dial back on her chess activities as she starts the strenuous two-year International Baccalaureate program at school.

Finally, here are some additional photos from the trip:

Flickr API error: Photoset not found

1Password for Teams and Families incompatible with VPNs

One of the services for which I’ve truly been happy to pay is 1Password for Families, which allows my wife and I to centrally manage information vaults that are shared among ourselves, and among our kids, across all our Mac and iOS devices.

Some time ago, I wrote about how I secure our home network with a VPN. After doing that, we began having to frequently respond to CAPTCHAs when accessing any website that uses the CloudFlare security platform, as CloudFlare (understandably) doesn’t trust the IP addresses of the Private Internet Access VPN service that we use. This is an annoyance, but certainly something we can live with.

Unfortunately, however, I recently discovered that all of our 1Password applications (iOS and Mac) have stopped syncing their data with 1Password’s servers. And to make matters worse, the apps don’t provide any feedback to the user that synchronization has failed! It was only after removing a Families account from one of the devices, and trying to add it back did I finally see a “No response from server” error.

My experience with CloudFlare-managed websites immediately let me to suspect that 1Password had their client API sitting behind CloudFlare, and an email to 1Password support confirmed this:

After reviewing the situation with his colleagues at 1Password, however, he then followed up to say that, sorry, but it looks like their service is just incompatible with Private Internet Access:

Right now, because so few users are affected by this, 1Password’s response is just: “Sorry, you can’t use our service if you’re going to use a VPN.” This seems short-sighted for the following reasons:

The problem doesn’t only affect users on Private Internet Access IP addresses. It affects users on any IP address that CloudFlare distrusts. Currently that’s at least PIA users, and almost certainly includes other popular VPN providers. But over time, one can certainly expect that set of IP addresses will expand.
More fundamentally, when accessing a website, CloudFlare provides a means by which a legitimate user on a distrusted IP address can successfully get through—by responding to a CAPTCHA. In other words, there’s a model in place by CloudFlare that anticipates false positives. If you’re going to put your software API in front of CloudFlare, as 1Password has done, then you must also engineer a model and user experience that accounts for false positives. (Perhaps CloudFlare offers a mechanism to surface a CAPTCHA like mechanism to the human user of an app that’s getting trapped on its API by CloudFlare.)

Hopefully, the team at 1Password will reconsider the situation, and find a solution.

How to manage a Tomato router via the CLI using Keyboard Maestro

As I wrote about a few weeks ago, I have my home network connected to the internet through a VPN router, running the Tomato firmware. Although the setup works great, I did run into two issues which I needed to detect and resolve programmatically, using Keyboard Maestro (KM):

Rebooting the router

The router frequently hangs—about once every few days—and requires a reboot. Manually logging into the web interface to click the “reboot” button gets tiresome, and so I decided to see whether I could automate this with Keyboard Maestro.

I have KM running on a Mac mini whose ethernet interface is connected to my VPN-protected LAN, and whose wifi interface is connected to my ISP’s router. The wifi interface is configured as default in the Network Settings preferences such that all internet traffic is, by default, routed through the ISP’s router. (This is to provide Slink-based remote access to my home network.)

So the first problem to solve was how to test internet access on the non-default ethernet interface? Fortunately, the gracious KM author, Peter Lewis, discovered that the ‘ping’ command supports an option (‘-b’) to specify the network interface.

Now that I could check if the router was down, the next problem to solve was programmatically rebooting it. The Tomato software, being a Linux distribution, supports SSH access, and Peter pointed out that if I install my SSH keys on the mini, KM could then login to the router without a password. That, and a little Googling, allowed me to figure out the KM text script needed to reboot the router via SSH.

Putting this all together, here’s the KM macro (configured to run every 5 minutes) I created to test if the Tomato router is down, and reboot it if so. (It’s configured to run every 5 minutes.)

Now, you might be wondering what the ROUTER_REBOOTING variable is for. Turns out, there’s another Tomato-related issue I also solved with Keyboard Maestro.

Restarting the router’s VPN client

The Tomato router supports two VPN clients, VPNClient1 and VPNClient2. I have client 2 connected to a US-based VPN server, and route my AppleTV through that, allowing me to watch content that is IP-restricted to the USA. For minimum latency, though, I have client 1 connected to a server in France, and have it configured to route all other traffic on my home network.

Problem is, when the router boots, and perhaps due to the order in which the two clients start, all traffic ends up getting routed through US-based client 2. To fix this, I just need to stop and restart client 1.

To address this problem, I created another KM macro that that checks the geo-location of my external IP address, and if it’s not “FR”—and if the router isn’t currently rebooting; hence the ROUTER_REBOOTING check—then it restarts the VPN client 1.

Lance is the 2016 Spanish National U14 Chess Champion

Between July 11 and July 16, the 2016 U14 Spanish National Chess Championship was held at the beautiful Best Western hotel in Salobreña, Spain.

Lance won this category in 2015 as a U12 player, and so although it was his first official year in the U14 category, he actually came into the tournament as the defending champion, and was ranked first by ELO rating among the 151 participants.

After nine rounds of classical play, in which each game can last four hours or more, Lance finished with 7.5 points out of a possible nine, tied with Marcos Lianes and Salvador Guerra. Lance had the highest tiebreak—the average performance rating of one’s opponents—and therefore finished first, to become the 2016 National Champ!

https://www.flickr.com/photos/matthenderson/28106238180/

On the weekend after the tournament, as is customary, the rapid-chess championship was also held, with nine rounds played over the Saturday afternoon and Sunday morning. Going into the last round, Lance was tied for the lead with Gabriel Quispe. The two played to a draw, and was joined by Marcos Lianes, who won his last game, to finish the tournament tied with 7.5 points out of nine.

In the rapids tournament, however, the tiebreak wasn’t in Lance’s favor, and he finished third behind Lianes and Quispe.

https://www.flickr.com/photos/matthenderson/28106238720/

The final results are posted at Info64:

All in all, a great tournament, and now it’s on to play the U16 championship, which starts this week.

My awful experience installing Windows 10 in VMWare Fusion 8

In reviewing Lance’s performance at the Spanish national championship this past weekend, the GM trainer from Andalucia strongly encouraged us to buy “ChessBase” as a tool to keep up with the latest in opening theory. Since Lance already runs Windows 7 in VMWare Fusion—in order to run PlayChess and TeamSpeak—I didn’t expect there to be any issues installing ChessBase (which is only available for Windows.)

I was wrong. Trying to install ChessBase in Windows 7, I got an error that some C++ runtime was missing. I downloaded the runtime from the link included in the error message, but it wouldn’t install either.

Not wanting to waste time on all this, I figured the best way forward would be to just update to the latest Windows—i.e. Windows 10. And so began the following nightmare:

When you go to the Microsoft store to buy Windows 10, you’re presented with three options—(1) Free upgrade for Windows 7, 8, or 8.1 (2) Buy Windows 10 (Download), and (3) Buy Windows 10 (USB – English). (I’m not sure why “English” is listed on the USB option…)
Here’s what you see when you click the free upgrade option—a screen that suggests you buy a new PC, and provides zero information about how to upgrade. Heavy sigh, but having to jump through hoops to get something free didn’t strike me as surprising.
Again not wanting to waste time, I decided to just buy the thing. And the purchase process turned out to be a lot more straightforward than the free upgrade process, as expected.
After my purchase, I had to choose which version to download: Windows 10, Windows 10 N, Windows 10 KN or Windows 10 Single Language. Of course, there’s no explanation of what the differences are, so I just rolled the dice and chose the first.
Then you have to choose “Home” vs “Pro”. Again, no explanation of the differences, so I just chose “Home”.
Then you have to choose 32-bit or 64-bit. You’d think Google could help with this, but not really. Rolling the dice again, I just went with 64-bit. Bigger is better, right?
I was then given a download link to an .iso file, and product number. I downloaded the .iso file, and used it to start the process of creating a new VM in Fusion 8. Fusion asked for the username, password and product number—all of which Windows later asked for again.
When the Windows 10 installation window opened, it asked for the product number. I entered mine, and was told the number was invalid. Of course. After a bit of Googling, I learned that you actually don’t need a product number to install Windows 10 (Was my purchase for nothing?) so I clicked, “I don’t have a product number”.
The next screen asked if I want to do an “Easy Install” or a “Custom Install”. According to Google, one shouldn’t touch the Custom Install!
Clicking “Easy Install” led me to a screen saying that I’d booted my Windows machine from “Windows Installation Media”, and that I needed to disconnect that, reboot windows, and then re-insert the media when prompted. WTF!?! Now, you would think that somebody else would have ran into this, and you’d also think that VMWare Fusion themselves would have run into this while installing Windows 10, but the internet offers no solution to this problem.
In desperation, and feeling I’d hit a complete dead end, I decided to give the dreaded “Custom Install” a try. I clicked that, surprisingly wasn’t asked to make any custom choices, and the Windows 10 installation proceeded to complete successfully. Un-believ-able.
In order to get reasonable integration with your Mac, the first thing you have to do when a new VM boots is install “VMWare Tools”. Unfortunately, for me, the “Install VMWare Tools” menu item was grayed out. Google said the problem is that VMWare Tools requires a virtual CD-ROM device to be attached. (Why on earth?!?…) Unfortunately, in my case, there was no way to add a CD-ROM to the VM, because neither my MacBook Air nor Lance’s iMac have a physical CD-ROM! Trying to add one anyway using the “Auto-Detect” setting led to a boot error, “Can’t attach to the Sata 0.0 device”. And again, unthinkably, neither the VMWare website nor Google could seem to help!
The solution, as I eventually discovered, was to manually download VMWare Tools (which of course comes with no README; just a bunch of .iso files), attach the Windows 10 VM’s CD-ROM device to the “Windows.iso” file included with the VMWare Tools manual download, boot the VM, and then install VMWare Tools manually from the attached “virtual CD-ROM”. Apparently, this was only needed on the first installation of VMWare Tools, and that in the future it’ll be able to upgrade itself without a virtual CD-ROM attached. We’ll see…

At this point, almost five hours later, I could finally install ChessBase under Windows 10, and provide it access to our shared network device.

To me, it seems absolutely crazy that it hasn’t occurred to anyone at VMWare to write up a tutorial documenting what I imagine is a common use case of someone wanting to purchase Windows 10, and then create a Fusion VM, with VMWare Tools installed.

Update—After posting this article, a couple other observations came to mind, illustrating just how crazy this Windows world is:

When you install MacOS, you’re shown a progress bar. The progress might not be accurate, but at least you’re shown the visual indication that something is happening. When you install Windows 10, you get a screen that shifts between dark and light blue (is it breathing?) and says, “We’ve got some great features waiting for you.” It’s not really clear that something is going on in the background. In fact, at some point, I clicked the screen just to make sure it wasn’t waiting for me to do that to continue!
The biggest hilarity happened when installing ChessBase. The first time you launch the app, it asks you to enter its product code. That’s normal. What’s not normal, though, is that it also asks you to respond, on the same screen, to a CAPTCHA! Can you imagine? An installer with a CAPTCHA! But it gets worse. All the letters in the CAPTCHA are capitalized, and the input field auto-capitalizes whatever you type in, which, OK, seems to make sense if they want to remove case-sensitivity from the operation. But here’s the thing—if you type in a lowercase letter, even though it gets upper-cased in the input-field, the lower-case letter gets sent to the validation, and IT IS case-sensitive! So even though it looks like you’re submitting an upper-case letter, you’re not! Insane!

SendGrid made things right

Update — Readers will note that I’ve changed the title and URL of this article, and that’s because shortly after posting it, representatives of SendGrid reached out, apologizing for the situation, explaining that my situation isn’t what they intend, and offering to make it right.

All-in-all, barring what happened this morning, we’ve always had good experiences with SendGrid, and their product is really well designed, and so I’ve decided to continue giving them our business.

My company Makalu was engaged by a US educational non-profit to develop an online platform called “Letters 2 President,” through which America’s youth can publish letters to the candidates of the 2016 presidential election. While the platform is under development, a website was established to inform about the project, and start taking preliminary signups from schools, libraries and other organizations wishing to participate.

http://www.letters2president.org

Most web applications these days outsource certain functions to third-parties. For example, it’s typical to use Amazon S3 for storage, CloudFlare for content distribution and site protection, and in the case of sending transactional emails, we’ve tended to use SendGrid.

Until now, that is. After today, we’ll no longer use their services, nor will we continue to recommend them to our customers. Here’s why…

For our project, we need to send notification emails to our customer whenever new applications arrive from organizations wishing to participate. We need to send notification emails to organizational administrators when group leaders create accounts. And we need to send notification emails to group leaders whenever a student creates or modifies a letter to be published on our site.

That’s why we need a transactional email service like SendGrid.

As usual in our projects, we create dedicated accounts with these third-party providers, as opposed to using our own Makalu accounts, so that when a project is finished, we can hand over everything—including provider accounts—so that the customer is free to operate their project without any dependencies on Makalu.

And in that regard, this morning I tried to setup a SendGrid account for use in our Letters 2 President project.

Ten minutes after creating the account, I received a notice from SendGrid that based on their review of a broad range of data points, our provisioning request had been rejected.

A rejection based on an automated data check process didn’t come as a surprise, for a number of reasons:

Although the account was created in the customer’s name, the email address I used when setting it up was a Makalu address, in order that, until project handover, we can receive all the various confirmation and related emails from the service.
As our office is located in Europe, the IP address that SendGrid saw on the request was outside the United States, and not corresponding to the business address specified in the account creation process.

I imagined that a simple email could clear the matter up, and so I replied to the rejection notice, explaining the purpose and nature of our project, explaining who’s involved, explaining the reasons for the checks I imagined triggered the rejection, and offering to answer any questions they might have in order to get the account provisioned.

Another 10 minutes later, I received a cold and unfriendly follow-up saying thank-you, but based on reasons that won’t be disclosed, our account will not be activated. Just like that. No chance of a discussion. End of story.

And, adding insult to injury, their note ends with the sarcastic-sounding, “We wish you the best in your future endeavors.”

I completely understand why a transactional email company has to be careful in the provisioning of accounts. We all know how big a problem spamming is. But I can’t understand at all why a company would be completely unwilling to even engage with a new customer who presents a clear case for the legitimacy of their use of the service.

So that ends any current and future business relations we’ll have with SendGrid. Fortunately there are many other providers of transactional email, who’ll perhaps enjoy the exposure when we later publish about the building of this exciting new platform.

How to switch wifi networks with Keyboard Maestro

In a recent blog post I explained how I secure my home network with a VPN. In that article, I also explained how I enabled external access to my home network, using the Slink software running on a Mac mini server, whose primary network interface is wifi connected to my ISP router, and second network interface is ethernet connected to my home gigabit switch.

This setup works great, but it did require solving a tricky problem:

My home wifi network (created by the AirPort Extreme) is called “Hacienda”, and the wifi network created by the ISP router is called “HaciendaOlive”. Since I want all my home devices connected to Hacienda, that network is given first priority over all other known networks on my iPhones, iPads, etc.

The problem is that that network priority list propagates to the Mac mini (and all my devices) via iCloud, and so anytime there’s a network interruption or the machine reboots, the Mac mini connects to the Hacienda wifi network (instead of HaciendaOlive)—which of course kills my external access to that machine.

What I need is that the mini, and only the mini, has HaciendaOlive set as its highest priority wifi network. But this doesn’t see to be possible, unless I’d be willing to disable iCloud on that machine.

My solution to this problem was a Keyboard Maestro macro which runs every five minutes, checking whether the computer is connected to the HaciendaOlive network, and if not, switching it to that network. This required researching some obscure AppleScript code, and so I thought I’d post the macro here for the benefit of others searching for how to switch wifi networks using Keyboard Maestro. The blurred text in the image, is the wifi network password.

Enjoy!

Milton Friedman on Greed

In this interview, Milton Friedman responds to Phil Donahue’s concerns about greed.

How to protect your home network with a VPN router

In this article, I describe how I added security to my home network by installing a router that directs all internet traffic through an encrypted VPN connection. The adventure includes my experience with the FlashRouters company, the Tomato router firmware software, an OpenVPN connection to the Cloak network, the Linksys E2500 router and the Netgear Nighthawk R7000 router.

Continue reading How to protect your home network with a VPN router