Splunk Management Port

The Splunk management port 8089 has come up repeatedly in enterprise wide security scans. I understand the security team's concerns: This port runs HTTPS with a self-signed certificate, which is a big no-no for any IT person who is even slightly security conscious.

Self-signed TLS certificate is not a security threat in and of itself. The main weakness is that, unless the Certificate Authority is known to clients connecting to the port, the clients are left to trust that certificate, rendering the connection prone to man-in-the-middle attacks.

To make a self-signed certificate as secure as a public CA signed one, Splunk just need to verify the self-signed cert using the signing CA cert. Or, even better yet, mutual authentication may be performed on these connections. However, that also means much stricter management of these certs.

My organization's security team suggested that we get third-party signed certificates for all the Splunk forwarders, which means we would need to obtain hundreds of certificates. That number could potentially grow to over a thousand or two. We are part of a large university with the benefit of largely self-administered certificate signing. But managing such a number of certificates is still a pretty big task unless the entire workflow of certificate creation, signing and renewal can be automated.

One remedy is to disable the port on all Splunk installations where the port is not absolutely needed. That means all the Universal Forwarders, whose main purpose is to collect data and forward to indexers. However, port 8089 is labeled a management port. Access to Splunk's REST APIs goes through that port. Disabling it means apps depending on the REST APIs, such as the UFMA app, may not function as designed.

The management port obviously exists for a reason. Beyond the REST API, do we lose any other functionality if we have to disable the port?

Splunk, SSO, Enterprise Security

Splunk is a powerful tool, therefore it is natural that the need to secure it in an enterprise is strong. With the push from upper management to secure access to privileged systems, 2-factor authentication (2FA) is the next defensive weapon to be deployed against attacks from both the outside and inside.

Looking into authentication and authorization mechanisms, especially SSO (single sign-on) in Splunk, it is amazing how much things have stayed the same since I first installed Splunk 4 back in 2011. This time, I am investigating the feasibility of using SAML 2 for Splunk to achieve both single sign-on and role-based access control (RBAC).

What I have found so far are not very encouraging and I seriously hope that someone prove me wrong.

Splunk has a number of ways for user authentication:

• Splunk internal: User information is stored in Splunk itself. A user would have to sign on each and every time visiting a different Splunk web host.
• LDAP: Splunk uses an LDAP server as user ID store.
• Proxy SSO: Splunk web must run behind a proxy server, such as Apache or IIS, which actually implements the sign-on part of SSO.
• Scripted authentication: Splunk calls a custom script to perform user ID acquisition and authentication.

 For user access authorization:

• LDAP: Using LDAP for authentication allows Splunk to map LDAP groups to Splunk roles for access control.
• Script: The script for authentication is also used to assign roles to the user.

Splunk does not seem to allow both SSO and RBAC simultaneously: The only mechanism for SSO is running Splunk web behind a proxy server. But in that case, it still relies on internally stored user information or LDAP group mappings to assign roles to an authenticated user, if this post in the Splunk Answers still holds true. Even if that changes, fronting every Splunk web service with a proxy creates an administrative challenge when you have many Splunk search heads running on both Linux and Windows, for example. Unless, of course, one uses something like Novel NetIQ or Citrix NetScaler. But those mechanisms introduce complexities of their own.

Scripted authentication would offer a way to implement both SSO and RBAC simultaneously if Splunk passes along HTTP request headers to the script. But that does not seem to be the case from reading their documentation.

Good-Bye, at&t!

I strongly disagrees with at&t on the "Weev" case based on this Wired report (AT&T Hacker 'Weev' Sentenced to 3.5 Years in Prison). If the report is not proven wrong, I am going to say good-bye to at&t.

Maybe for good this time.