iCloud syncing of Mail.app rules and Smart Mailboxes

Setting the context

I run Mail.app on an internet-hosted Mac mini, in order to provide server-side spam filtering and to pre-process my email based on an extensive set of rules.

Many of those rules involve the application of tags to messages, via the wonderful MailTags plugin.

On the MacBook Air that I use at home and at work, I have a simple set of four rules that apply account-specific colors to incoming messages. On the other hand, that MBA does contain an extensive set of Smart Mailboxes, most of which search for messages containing certain tags.

Problems archiving and deleting inbox messages

I’ve been having awful problems recently with Mail.app’s ability to delete and archive messages in my inbox. I’ll archive a message, only to see it return 10 minutes later. I’ll archive it again, and it’ll again return sometime later.

If I then view my inbox on an iPad, however, I’ll see four copies of that message in the inbox! If I then delete them all from the iPad, the pesky original message will then be removed from the inbox on the Mail.app.

This has been driving me crazy!

An aside—Why am I seeing four copies?

Mail.app on OS X has an annoying behavior of only showing you one copy of duplicate messages; so in fact, there are four copies of the message in the inbox, but Mail.app is only showing me one.

That’s how four messages somehow get created. The first time I archive the message, a second copy gets created. The second time I try to archive, two additional copies get created.

Disappearing Smart Mailboxes & Rules

In an attempt to fix this irritating behavior, I uninstalled the MailTags plugin, which is currently in a beta version for OS X Mavericks. Most probably MailTags has nothing to do with it, but since it’s beta software, I thought I’d give it a try.

After uninstalling MailTags on my MacBook Air and launching Mail.app, I observed my entire list of Smart Mailboxes simply disappear—replaced by a single Smart Mailbox, “Today”. Re-installing MailTags, all my Smart Mailboxes returned.

After uninstalling MailTags on the mini and launching Mail.app, I opened the preferences to find all my rules gone, and replaced with the four simple rules I have on my MacBook Air!

So, obviously there’s some syncing going on, and obviously MailTags is somehow involved as well.

What’s going on here?

I emailed the folks at MailTags, and learned that MailTags keeps its own local database of Smart Mailboxes and Rules, so that if you launch Mail without MailTags installed, no rules or Smart Mailboxes will get created that contain tag-related rules.

Ok, that explains why things change when MailTags is uninstalled.

What’s strange, though, is that before uninstalling MailTags, I’d removed all the tag-related criteria in my various Smart Folders, and still they all disappeared after uninstalling MailTags. So, evidently, once a Smart Mailbox (or rule) makes its way into the MT local database (because of tag-related criteria), it’s not moved to the local Apple database if you later remove those tag-related criteria.

iCloud’s role in the mess

How is syncing entering the picture?

I did some research and learned that if you have Documents & Data enabled in iCloud, any Mail.app rules and Smart Mailboxes you create will get synchronized across all your Macs.

That’s terribly undesirable behavior for someone like me, who specifically needs different sets of rules and Smart Folders running in Mail.app on each of my Macs. And according to this article the only way to stop this behavior is to disable Documents & Data in iCloud.

I was going to do just that, when I saw 1Password in the list of apps using Documents & Data—and that’s the one app that I need to sync data across all my apps. So unless I’m willing to move 1Password syncing back to Dropbox, it looks like I’ll be stuck with syncing of Mail rules and Smart Mailboxes.

Unless… I keep MailTags installed. Remember, when MailTags is running, Mail.app’s Smart Mailboxes and rules are created from MailTags local database, which is not synced.

But with MailTags running, I’ll not be able to figure out whether it’s playing a role in my inability to delete/archive certain messages from my inbox—which is where this all began. So I’m kinda stuck.

My wish for Ulysses — Dropbox integration

Update: Sometimes blogging really pays off. Not 15 minutes after posting this article, someone on Twitter tipped me about a solution—In some recent release of Ulysses, they added the ability to add “external sources”. Using this feature, which is unfortunately only accessible via the small plus icon in the lower left of the screen, you can choose any folder as a document repository, and that works just fine with my Dropbox Notes folder.

Like many people these days, I have a “Notes” folder in Dropbox that acts as the central document repository for a variety of writing and editing workflows on my Mac, iPhone and iPad mini.

For long-form writing on the iPad, I use iA Writer, which reads from and writes to my Notes folder in Dropbox. For creating quick notes on the go using my iPhone, I use Simplenote, which synchronizes with nvALT (Notational Velocity ALT) running on my Mac. nvALT, in turns, keeps everything in sync with my Dropbox Notes folder.

While nvALT, on the Mac, is great for creating quick notes and searching my Notes folder, it’s not a particularly great environment for authoring in Markdown. For that, I prefer to use Ulysses III.

Ulysses is a great environment for authoring in Markdown. In addition to a quiet “focus” mode that’s common among writing apps these days, the killer feature of Ulysses for me are convenient visual controls for constructing those Markdown structures people like me tend to forget, like links and inline images. And those same controls result in a document that’s far easier to read, as meta-text like URLs are hidden from view.

But as much as I love Ulysses, it has one short-coming that, for me, is almost a show-stopper—it doesn’t support Dropbox as a document repository. So to work in Ulysses, I have to copy and paste text from a document in nvALT into a document in Ulysses, which introduces the risk that when I later move to the iPad to work, I won’t be working on the latest version

So my wish list for Ulysses contains only one item—Dropbox integration. And if—heck, let’s be optimistic and say, “when”—that happens, it’ll become one of the core apps running on my Mac.

Pausing the Cloud

We have three Macs at home, which run an assortment of cloud-backup and synchronization apps—including Dropbox, CrashPlan, BitTorrent Sync and Arq.

Whenever any one, or combination of, these apps saturates the ADSL upload bandwidth, downloading then becomes very difficult. And that presents a big problem when trying to do things like watch content on iTunes, or have a conversation over Skype.

These apps all try to address the problem, but in collectively (and sometimes individually) ineffective ways:

  • They all provide for bandwidth limitation through configuration. That doesn’t help, though, when you have multiple apps trying to upload at once. And who really wants to limit upload bandwidth during the 95% of the day when it wouldn’t matter anyway, just to address the 5% of the time when it’s needed?
  • Some provide for automatic bandwidth reduction when detecting that the network—on the given computer—is busy. But this doesn’t help when you have the apps running on multiple machines on the network.
  • All provide for manual pausing. Only Arq provides a means of pausing through scripting (or via a command-line interface); the rest require manually clicking their menu-bar icons. Only Arq and CrashPlan allow you to specify the amount of time to pause; for the others, you have to remember to unpause them. And BitTorrent Sync’s menubar icon doesn’t even have a “paused” state (like Dropbox’s), which nearly ensures that an unfortunate event—like realizing the file you’re working on isn’t up to date—will be the thing that reminds you that it’s not running.

Although I’d love to find a single-interface, elegant solution to the problem, life right now would be much easier if:

  1. All such apps provided a means of scripted pausing. That way, using my iPhone while sitting on the couch, I could connect to Keyboard Maestro and run a macro that would pause every app on every machine on the network.
  2. All such apps provided timed pauses. That way, I wouldn’t have to remember to unpause them. And for those that don’t provide timed pauses, for goodness sake, at least provide a paused state of your menu-bar icon.

iPad is Disabled

A few weeks ago, as I reported here on the blog I opened my iPad mini and saw, “iPad is disabled”.

In this state, the iPad can’t be used until it’s connected and sync’d with iTunes on its rightful owner’s Mac; a major pain if you happen to be traveling with nothing but the iPad! But this behavior is a safety measure by design, as “disabled” is the mode into which an iPad goes after one too many incorrect unlock PIN codes have been entered.

Wondering what could cause this, my suspicion was that somehow my iPad was getting frequently switched on and off while being transported in my bag. Why did I suspect that? Well, I’ve also recently noticed that while running with my iPhone in a fanny pack, I’ll sometimes hear what appears to be telephone dialing coming through the headphones over the music. When that happens and I take the phone out of my pack, I’ll find it displaying the “emergency call” screen. So somehow, just the friction of the fanny pack material against the face of the iPhone, while running, causes the iPhone to switch on.

But what could cause the iPad in my bag to switch on in a similar way?

I’ve noticed that if you twist the Smart Cover of an iPad mini in the same plane as the device, you can hear it switching on and off. So it’s not only separation of the cover from the device that activates it; but also disalignment of the cover against the device as well. (I suppose there’s a magnet in the cover whose specific position relative to the device is monitoried.)

My iPad has a Smart Cover, and when transported I slide the whole thing into a neoprene ACME MADE iPad mini sleeve. So my suspicion was that somehow, the device inside the neoprene sleeve, while being transported in my admittely tightly-packed Tom Bihn Ristretto bag was causing the Smart Cover to twist just enough to repeatedly activate and deactive the device until the “iPad is Disabled” state was engaged.

But around the same time as this was happening, something else frequently happened as well. I’d sometimes open the iPad and find the brightness control enabled, and pulsing up and down as if the light sensor had gone mad. I thought that, with a bit of imagination, that might also be related to whatever was causing the disabled mode.

I decided to go see Apple, and they seemed convinced it was a hardware problem and immediately replaced the device.

I thought the problem was resolved…until yesterday, when I pulled my iPad mini out of its bag and sleeve, opened the cover to once again find, “iPad is disabled”.

So now I’m back to the theory that this is being caused by twisting action on the Smart Cover, somehow happening when my iPad is being transported in its sleeve and bag.

I could try disabling the Auto-Lock feature. Having to manually press the button to turn it on wouldn’t be to bad. What’s bad, though, is the muscle-memory I have related to the action of just closing the cover to switch the device off. I’m pretty sure if I disabled Auto-Lock, I’d often close the cover, and then return later to find the battery completely depleted.

And that’s where are today—still a mystery. Hopefully somebody with a solution will stumble across this article!

Fixed — Daylite can’t connect to server after Mavericks upgrade

I run Daylite Server on an internet-connect Mac mini, and after upgrading to Mavericks (and OS X Mavericks Server) neither Daylite (running on my MacBook Air) nor Daylite Touch (running on my iOS devices) could connect to the server.

I noticed that if I disabled the firewall on the server, Daylite could then successfully connect. I inspected the server’s Firewall Options, and saw “Allow Incoming Connections” green on the Daylite Server Admin app, as well as the handful of Daylite-related server daemons.

I tried deleting those entries from the firewall and restarting the Daylite Server—hoping to see the familiar “Do you want to allow incoming connections?…” dialogs. But after restarting the Daylite Server those dialogs didn’t appear, and I still couldn’t connect.

As a last resort, I tried restarting the mini itself. Upon restart (and upon reconnecting via Screen Sharing), I finally saw the handful of Daylite daemon-related “Do you want to allow incoming connections?…” dialogs, and I confirmed them all. After that, Daylite could then successfully connect.

So my suspicion is that the first time the mini started after the Mavericks update, those dialogs appeared, but since I wasn’t connected to confirm them, they were declined by the OS. Which, if true, is strange, because there’s an option in the firewall that’s enabled, which should automatically allow incoming connections from “signed” apps. Could it be that Daylite Server, or some of its daemons are unsigned?

Anyway, problem now results, and I just wanted to post these notes for the benefit of anybody else that runs into this.

My iPad mini seems to have a hardware problem

My iPad mini seems to have a hardware problem. Frequently, when I activate the device by opening its Smart Cover, I find it in one of two states:

State 1: Possessed

In this state, the brightness control is visible, and pulsing up and down, on its own without any input from me, until it finally settles in the completely down position. Here’s a video:

Although I can authenticate with my pin code and use the device, the brightness control remains fixed on the screen and can’t be dismissed without a full power cycle of the iPad. (And, in this state, the brightness can no longer be manually controlled.)

State 2: Disabled

The other state in which I might find the device is Disabled.

This is the state the device gets in after too many failed logins. And if “Find my iPad” was previously enabled, you can’t recover from this mode through normal means. For example, if you try “Restoring from Backup”, iTunes will report that you must first unlock the device with your pin code—something that’s obviously impossible to do. Furthermore, logging into iCloud and removing the device from “Find my iPad” doesn’t help either.

When the device is disabled, you have to boot in Recovery Mode—by pressing the power-home button combination until the device restarts, and then continue holding the home button until it says “Connect to iTunes”—and then restore it from scratch, from iTunes. This requires a full download of the iOS software over the internet.

This—the device getting disabled—has actually happened to me twice now, and in both cases there was some mystery around its recovery.

  • The first time, I booted into Recovery Mode, started the restore process, and went to have lunch. When I returned, expecting to find the iPad in a “new” state and requiring a restore from backup, I instead found it completely up to date, with all my apps and contents. It was as if nothing had happened. Given my certainty that I’d need to subsequently restore from backup, I actually wondered whether the restore process had happened at all, or whether the device had somehow just recovered.

  • The second time, I was in the car and found it disabled while at a stoplight. (Yes, I was going to check email at a stoplight.) When I got home, the problem had cleared up on its own.

Visit to Apple


p>This afternoon while at the office, I opened the iPad to discover it in the “Possessed” state, and decided to immediately drive over to the local Apple Store, five minutes away, to have a Genius look at it.

The folks at the Apple Store, though, wouldn’t let me speak with a Genius, because I didn’t have an appointment. The problem, I explained, was that by the time I got an appointment, I would be left having to describe the problem to them, while right now, I can actually demonstrate it! Unfortunately, that wasn’t a good enough reason to break protocol, so they gave me an appointment for next Tuesday.

At this point, I can (a) leave the device as-is, without using it until next Tuesday in order to demonstrate the problem, or (b) reboot it, so that I can continue to use it, and then hope they’ll be able to investigate the problem next week with just a description (and perhaps the above video.)

Update: 2013-10-29 Replacement

Had my appointment at the Apple Store Genius Bar this morning, and given the combination of both a brightness anomaly and frequent “iPad is Disabled” occurrences, they replaced the iPad mini on the spot, saying that my old one almost surely had a motherboard problem.

Photo management — from Aperture to a file system and Everpix

Update — Since writing this article, Everpix has gone out of business, and I’ve moved to Apple’s iCloud Photo Library ecosystem.

I recently migrated my photo management workflow from one based on Aperture, to one based on a file system and the online service, Everpix. This article describes the whys and hows.

Continue reading Photo management — from Aperture to a file system and Everpix

Receiving email for virtual users in OS X Server 10.8

I receive my own email on a Mac running OS X Server 10.8, and just resolved a problem related to the creation of a second email user on that machine.

Let’s call the domain on which I receive mail on that machine, mydomain.com, which is completely different than the machine’s host name, which we’ll call, mymac.com. (The machine knows to receive email on mydomain.com since I added that domain in the Mail settings using the Server.app.)

This afternoon, I wanted to move my wife’s email to that machine as well.

Using Server.app, I created a new “services only” user, user2, and then setup her Mail.app with a new IMAP account pointing to [email protected]. That worked fine; Mail.app could login, and successfully manage IMAP folders.

The problem — The problem I ran into is that the Mac would not receive email on the address, [email protected]; it rejected all incoming messages with the error, “User unknown in virtual alias table”.

The solution — Thanks to this artice, I discovered that to setup a new mail account in OS X Server, it’s not enough to just create the user account, user2. You also have to right-click on the user name in the Users panel, and access the Advanced Options… where you’ll find an Aliases field, in which you have to enter the specific email address(es) on which you want that user to receive email. In my case, this was [email protected].

With that in place, the Mac then successfully began receiving mail.

Weekend at the computer—Dropbox, BitTorrent Sync and unencryptable drives

The recent news about the NSA’s access to the data hosted with cloud providers got me re-thinking how I store and access my sensitive documents—and not only from the point of view of privacy, but also in terms of convenience.

Continue reading Weekend at the computer—Dropbox, BitTorrent Sync and unencryptable drives

How to move folders that are synced with BitTorrent Sync

BitTorrent Sync is an app/technology that allows multiple computers and devices to share synchronized folders.

On my computer, I setup a folder called BTFamily, and synced that to my wife’s and kids’s computers. It worked so well, that I later added a folder shared between just my wife and myself, called BTMain.

At this point, the home directory on my Mac contained:

  • BTFamily
  • BTMain

Thinking that I might end up with quite a few synced folders, I was hoping to move those two folders into a single top-level folder called BTSync.

  • BTSync
    • BTFamily
    • BTMain

This article in the BitTorrent forums explains how to do this:

  1. Copy the folder’s “Secret”
  2. Remove the folder from SyncApp
  3. Copy/move the entire folder contents to a new location
  4. Re-add the folder’s new location to SyncApp, using the secret you copied in Step 1

SyncApp will still re-index the folder, but it will then carry on sync’ing with the folders on the other devices you’d previously given the same key to. You won’t have to re-issue a new key to your 10 other people!

Using a Contacts group in my email ReadLater rule

I get a lot of email that I’m interested in reading, but which I don’t want cluttering up my inbox. In the past, I tried creating a mail rule that would immediately archive mails (keeping their status unread) with a sender address matching those within a long list contained in the rule. That got unwieldy after a while, since Mail.app’s rules definition interface wasn’t really meant for managing very long lists of conditions, and so I abandoned the rule.

Recently, though, I had a better idea — adding all theses senders to a “ReadLater” group in Contacts.app, and then having my Mail.app rule contain a single condition—i.e. checking whether the sender is a member of that group. Works great, and easier to identify just those mails in my Smart Folder for later reading.

Switched to Parallels and then back to VMWare

Update: Since publishing this article, VMWare reached out and resolved the situation in a way that made me very happy. Be sure to see the details about that at the end of the article.

In March of this year (2013), some colleagues of mine in Berlin called to ask if I could purchase a copy of VMWare Fusion for Mac for them, because the VMWare website would not allow them to purchase the software. Each time they’d try to make the purchase, they’d get an “authorization failed” error.

Continue reading Switched to Parallels and then back to VMWare

Proposal for a centralized credit card data access service

Having received a new American Express card, I just finished updating the billing details at the thirteen online services which periodically charge that card.

And it took much longer than I would have imagined!

Of the thirteen services, I experienced thirteen different workflows for updating my billing details. At some, I simply had to update my card’s expiration date year. At others, I additionally had to update the security number. And at yet others, I had to completely delete the old card, and add a new one. And incredibly, a few sites presented forms which disabled pasting into the credit card number field—forcing me to carefully type 15 numbers—aaarg! Finally, at site’s like AT&T Wireless, it took forever to even find where to update my billing information.

There’s got to be a better way, which got me thinking…

I wish a service existed where I could store my credit card details, and then provide oAuth access to the various online services I use which need to charge my card. In the same way that I give services like Instagram oAuth access to my Facebook and Twitter accounts, I’d give services like Amazon AWS and Dreamhost oAuth access to this centralized credit card data service.

Such a service, if it existed, would provide several benefits:

  • When I need to update the expiration date and security code of my credit card, I’d only have to do it once, in one place.

  • If I decided to switch from using an AMEX card to using a VISA card, I could update the card in use at all my subscribed services from one place, with a single action.

  • In terms of enhanced security, the service would be implemented such that my authorized online services could charge my card, without actually getting access to the card details. (No idea how that’d work, but I’m confident it would be possible.)

  • Such a service would make it easier for me to actually remember all the different services that are presently authorized to charge my card. I’d review that list from time to time, and revoke access to those services I no longer use.

I’d be surprised if such a thing doesn’t exist, but I suppose it doesn’t since none of my online services like Amazon AWS and Dreamhost provide for any other payment method than entering my credit card details in their own systems (or the systems of their payment processors).

Google Authenticator broken in iOS7. Use Authy instead.

I discovered this morning that the Google Authenticator app is broken in iOS 7. While it does display authentication codes, it doesn’t display the associated service description. So unless you only use Authenticator for a single service, you’re kinda stuck.

Google are aware of the issue, but haven’t yet released a fix.

Urgently needing to login to Amazon AWS, I discovered an alternative app called Authy that does work on iOS 7. I was quickly able to get setup with (the better looking) Authy and Amazon AWS, Google and Dropbox, and subsequently deleted the broken Google Authenticator app.

The downside for developers of automatic app updating in iOS 7

Manual app updating, prior to iOS7, was beneficial to developers as a mechanism to notify their customers that a new version was available1. Since Apple doesn’t otherwise provide developers with a way to communicate directly with their customers, that indirect mechanism served an important role.

in iOS 7 apps silently get updated, automatically. This morning, I just happened to wander into the App Store app and discovered several apps that had recently updated—some of which contained interesting functional additions.

Perhaps with iOS7, enabling customer communication by finding creative ways of capturing user email addresses—i.e. ways that don’t violate Apple’s policies—will take on a higher priority with developers.

  1. I think some developers abused this with unnecessarily frequent updates, simply to trigger a reminder that their app exists. 

Discovered and fixed a huge CrashPlan log file

A couple of days ago, I discovered that my CrashPlan log (in /Library/Logs/CrashPlan/engine_output.log) had gotten huge—over 60GB. Googling, I found this article that describes how to increase CrashPlan’s memory. In cases where you have CrashPlan backing up millions of files, it can be necessary. After increasing the app’s memory, and deleting the log file, it has since remained at a more reasonable 1.5GB.

Here are the steps:

  1. Launch a terminal window

  2. Shutdown the crash plan daemon. You will be asked for your admin password. Go ahead and enter it here.

    sudo launchctl unload /library/launchdaemons/com.crashplan.engine.plist

  3. Edit the plist file with BBEdit:

    bbedit /library/launchdaemons/com.crashplan.engine.plist

  4. Find Xmx512m and change it to Xmx1024m. This will allocate 1024mb of memory instead of the default 512mb

  5. Save the file and quit BBEdit

  6. Relaunch the Crashplan daemon (corrected Apr 19th 2012)

    sudo launchctl load /library/launchdaemons/com.crashplan.engine.plist

Orphaned Facebook apps

We have a customer in Germany for whom we developed several Facebook apps used in their business. The apps were associated with the personal Facebook account of our customer contact.

Several months ago, without notification, Facebook converted this person’s personal account to a business account. As a result, all of their apps became orphaned—no longer associated with the account, and no longer accessible for modification.

Facebook later began sending notifications to us (and all app developers) about required app updates vis-a-vis Facebook’s new privacy policies, and warning that non-complying apps would be deactivated.

But without access to the apps, we obviously can’t make any updates.

We’ve tried everything we can think of to get in touch with someone at Facebook about this, but have been unsuccessful. We’ve tried posting in Facebook-sponsored developer forums, posting to several Facebook-related twitter accounts, etc.

As far as we can tell, Facebook does not expose a contact address for developers who have problems that need resolution. And so we are stuck with a set of orphaned apps that we can’t access, but which our customer needs in the operation of their business.

I’m posting this to my blog in the hopes that someone might stumble across the article, who could help. If you can help, please contact me here or at matt at makaluinc.com. Thanks so much.

iCloud syncing of Mail rules

I have a personal account setup on three different Macs—my own MacBook Air, an iMac that serves as our home server and the kid’s mini. In my account on the iMac, I had Mail running with an elaborate set of rules, including those related to spam handling.

This morning, I woke up to discover loads of unfiltered messages in Mail on my iPhone. I checked the rules panel in Mail on the iMac and discovered all my custom rules were gone.

Continue reading iCloud syncing of Mail rules

How to upload Skitch images to Amazon S3 using Dropzone

Like many people, I’ve long used Skitch to snap, annotate and share screenshots. Sharing involves Skitch uploading the image somewhere, and putting the destination URL on the clipboard for your immediate use.

In addition to Skitch.com, the original product allowed you to specify other upload destinations, including your own FTP or SFTP server. And over time, they added other upload options like WebDAV and Flickr.

Continue reading How to upload Skitch images to Amazon S3 using Dropzone

Daylite and Direct Mail for managing customer relations and communications

In Makalu Interactive, we use two Mac OS X applications—Daylite and Direct Mail—to manage customer relationships and communications, within a context of both client services and the selling of our own products.

Continue reading Daylite and Direct Mail for managing customer relations and communications