Twenty dollars to the first person who solves my OS X firewall problem

This article discusses a strange issue related to the OS X 10 firewall, mysteriously restarting itself, and disallowing traffic to applications that should be whitelisted.

Update 2014-03-06: SOLVED! (See below)

The OS X application firewall running on my Mac mini is preventing Daylite clients from remotely connecting to the Daylite server running on the machine. Unsuccessful at solving the problem myself, I’m offering $20 to the first person whose help results in resolving the issue.

To be more accurate, I’m actually observing two separate issues, and the resolution of either would solve the overall problem. So I’m offering the bounty to the first person who helps me solve either of them.

Background

I have a Mac mini running OS X 10.9.2, along with Mavericks Server. The mini is co-located with one of the leading Mac mini hosting providers. In addition, I have the latest version of Daylite Server running on the machine, to which Daylite clients (OS X and iOS) need to connect.

Problem 1: Daylite can't connect when the firewall is enabled

When the OS X firewall is enabled, Daylite clients are unable to connect to the server:

When I first installed Daylite Server on the machine, the OS popped up several notifications asking for permission for various Daylite-related processes to accept incoming connections. I accepted them all.

And, in fact, for quite some time, Daylite clients were successful in connecting to the machine. At some point, and I can’t remember precisely when, it just stopped working; as long as the firewall was enabled, Daylite clients were no longer able to connect.

I have tried removing all these processes from the firewall, and manually adding them back. Sometimes after doing that Daylite clients can connect, but after the next restart of the machine, they can no longer connect.

(I don’t know if it’s related, but for completeness, I believe this problem started around the time I tried installing, and later uninstalled, Ice Floor, a GUI to manage a separate OS X firewall called “pf”. Again, though, I did run the Ice Floor uninstaller, so I presume anything it installed/configured was undone upon uninstalling it.)

Problem 2: The firewall re-enables itself each night

One work-around to Problem 1 is to simply disable the firewall, and that’s what I’m willing to do in the absence of a real solution to Problem 1. But even that doesn’t solve my problem. Why? Because each night at midnight, something happens on the machine that re-enables the firewall! In the morning, I connect to the machine to find this:

So, I’ll connect to the machine during the day, unlock the Security & Privacy preference pane, disable the firewall and re-lock the preference pane. At that point, Daylite clients successfully connect to and sync with the server.

But then the very next morning, I find client connections again failing. I connect to the mini, to find that, as every day, the firewall has mysteriously been re-enabled!

I say that the re-enabling happens at midnight, but I’ve not actually connected to the machine to confirm it happens precisely at that time (because I’m in Europe, and the machine is in the United States). But I’m sure it happens around that time. I’ve checked the Console, however, and don’t find any messages posted around midnight that would seem related.

Bounty rules

As mentioned at the beginning, I’m offering $20 to the first person who successfully helps me resolve either of these two problems. (My preference is a permanent solution to the first, but either will do.) Please post your ideas/solutions as a comment on this blog post.

Only one person can receive the bounty—i.e. the first who provides me with the information that results in solving my problem. If two people post the same solution in the comments, the one arriving first will receive the bounty. If you need clarifying information, just ask in the comments, and I’ll try to clarify.

The bounty can be paid by PayPal if outside the United States, or ACH transfer if within the United States (via Capital One 360 P2P payment service).

Thanks so much, in advance!

Solved! (Problem 2, at least)

And the solution came from none other than Makalu’s own system administrator, Niall O Broin. (OK, with SSH access to the machine Niall has a slight advantage…)

What Niall discovered was the following entry in the crontab, set to disable and then re-start the firewall each night at 00:01 (i.e. midnight):

• /usr/libexec/ApplicationFirewall/socketfilterfw --setglobalstate off && /usr/libexec/ApplicationFirewall/socketfilterfw --setglobalstate on

What we don’t know, is how that entry made its way into the crontab, which is a Unix scheduler that Apple recommends against, in favor of launched.

(What’s particularly funny, and ironic, is that after having unsuccessfully spent about two hours searching for the cause, Niall discovered it when we concluded that, at least for the moment, the best course of action would be to add a crontab entry at 00:30 to disable the firewall each day. We though, if we can’t find the problem, at least we can hack a solution!)

Enjoy this article? — You can find similar content via the category and tag links below.

• Categories — Tidbits

Questions or comments? — Feel free to email me using the contact form below, or reach out on Twitter.