FR: Support reading service account key from preferences

[hjuutilainen]

First of all, thank you for this project!

I'd like to see a way to store and read the service account key from NSUserDefaults. This would enable distributing the key through configuration profiles and MDM. The key could be maybe be stored as base64 encoded blob in ManagedInstalls alongside main Munki preferences.

[waderobson]

Sorry for getting back so late. I think that would be a nice feature to add but would want to still be backward compatible with the json file. I actually got something together #9. Have a look and let me know if that works for you.

[securitygeneration]

I agree, this is a good idea, and I like the backward compatibility. Do you have some instructions about how to put the preferences in the right place?

[securitygeneration]

It would be cool to be able to push the config for GCS Auth via MDM profiles, like how it possible for S3:

[waderobson]

@securitygeneration I actually started on this but haven't had time to test it. If you could test it out that would speed this along. This is the version to use https://github.com/waderobson/gcs-auth/blob/cfpref/middleware_gcs.py

the preferences you need to set are GCSClientEmail GCSPrivateKey
the values would be obtained from the json file used in the current setup

[securitygeneration]

@waderobson do you have a recommendation for how to set these preferences via MDM (e.g. profile)?

For testing I tries setting these parameters locally using:

defaults write ManagedInstalls GCSClientEmail -string munki-sa@project.iam.gserviceaccount.com
(and another for the GCSPrivateKey)

and confirmed that they exist in CFPrefs by using this script: https://github.com/dataJAR/jnuc2016/blob/master/cfprefs.py

I put your middleware file above into /usr/local/munki and removed the private key file. This is what I get when running managedsoftwareupdate

Processing options through middleware
ERROR: Unexpected error in updatecheck:
Traceback (most recent call last):
  File "/usr/local/munki/managedsoftwareupdate", line 1218, in <module>
    main()
  File "/usr/local/munki/managedsoftwareupdate", line 921, in main
    updatecheckresult = updatecheck.check(
  File "/usr/local/munki/munkilib/updatecheck/core.py", line 82, in check
    mainmanifestpath = manifestutils.get_primary_manifest(client_id)
  File "/usr/local/munki/munkilib/updatecheck/manifestutils.py", line 175, in get_primary_manifest
    manifest = get_manifest(clientidentifier, suppress_errors=True)
  File "/usr/local/munki/munkilib/updatecheck/manifestutils.py", line 121, in get_manifest
    dummy_value = fetch.munki_resource(
  File "/usr/local/munki/munkilib/fetch.py", line 439, in munki_resource
    return getResourceIfChangedAtomically(url,
  File "/usr/local/munki/munkilib/fetch.py", line 394, in getResourceIfChangedAtomically
    changed = getHTTPfileIfChangedAtomically(
  File "/usr/local/munki/munkilib/fetch.py", line 526, in getHTTPfileIfChangedAtomically
    header = get_url(url,
  File "/usr/local/munki/munkilib/fetch.py", line 247, in get_url
    options = middleware.process_request_options(options)
  File "/usr/local/munki/middleware_gcs.py", line 115, in process_request_options
    options["url"] = gcs_query_params_url(options["url"])
  File "/usr/local/munki/middleware_gcs.py", line 108, in gcs_query_params_url
    url = gen_signed_url(file_path)
  File "/usr/local/munki/middleware_gcs.py", line 88, in gen_signed_url
    key, client_id = read_keystore()
  File "/usr/local/munki/middleware_gcs.py", line 78, in read_keystore
    key, client_email = _read_json_keystore()
  File "/usr/local/munki/middleware_gcs.py", line 57, in _read_json_keystore
    ks = json.loads(open(JSON_FILE_PATH, "rb").read().decode("utf-8"))
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/munki/gcs.json'

It seems to still look for the file? Or is it only doing that because your script can't find the prefs I've set?

[hjuutilainen]

I briefly tested this with a custom profile and using a payload content with something like this:

<key>PayloadContent</key>
<array>
    <dict>
        <key>PayloadDisplayName</key>
        <string>Munki Settings</string>
        <key>PayloadIdentifier</key>
        <string>com.example.munki.ManagedInstalls</string>
        <key>PayloadOrganization</key>
        <string></string>
        <key>PayloadType</key>
        <string>ManagedInstalls</string>
        <key>PayloadUUID</key>
        <string>72574D2D-7F35-4DE1-BCC0-8C95BE3BCCCF</string>
        <key>PayloadVersion</key>
        <integer>1</integer>
        <key>SoftwareRepoURL</key>
        <string>https://storage.googleapis.com/your-bucket-name-here</string>
        <key>GCSClientEmail</key>
        <string>service-account-name@your-project-name.iam.gserviceaccount.com</string>
        <key>GCSPrivateKey</key>
        <string>-----BEGIN PRIVATE KEY-----
redacted
-----END PRIVATE KEY-----
</string>
    </dict>
</array>

Seems to be working very well. Thank you @waderobson for this!

[hjuutilainen]

Would it make sense to store the private key as a base64 encoded blob (string) so that it's guaranteed to contain ascii characters only?

[waderobson]

I guess it won't hurt. I was just trying to keep it simple "copy and paste what was in the json file". You've worked with these mdm profiles more than I have so I'm happy to defer to you.

[securitygeneration]

I briefly tested this with a custom profile and using a payload content with something like this:

<key>PayloadContent</key>
<array>
    <dict>
        <key>PayloadDisplayName</key>
        <string>Munki Settings</string>
        <key>PayloadIdentifier</key>
        <string>com.example.munki.ManagedInstalls</string>
        <key>PayloadOrganization</key>
        <string></string>
        <key>PayloadType</key>
        <string>ManagedInstalls</string>
        <key>PayloadUUID</key>
        <string>72574D2D-7F35-4DE1-BCC0-8C95BE3BCCCF</string>
        <key>PayloadVersion</key>
        <integer>1</integer>
        <key>SoftwareRepoURL</key>
        <string>https://storage.googleapis.com/your-bucket-name-here</string>
        <key>GCSClientEmail</key>
        <string>service-account-name@your-project-name.iam.gserviceaccount.com</string>
        <key>GCSPrivateKey</key>
        <string>-----BEGIN PRIVATE KEY-----
redacted
-----END PRIVATE KEY-----
</string>
    </dict>
</array>

Seems to be working very well. Thank you @waderobson for this!

I tried doing this, but apparently the mobileconfig file isn't valid. Do you have a full mobileconfig file example @hjuutilainen ?

[securitygeneration]

Just wanted to follow up on this one. Any guidance on the mobileconfig would be appreciated!

[securitygeneration]

So, was able to get those params into a signed mobileconfig (and they appear correctly in Profiles), but the plugin bails out:

Jan 14 2021 13:53:11 +0100 ERROR: Unexpected error in updatecheck:
Jan 14 2021 13:53:11 +0100 Traceback (most recent call last):
File "/usr/local/munki/managedsoftwareupdate", line 921, in main
updatecheckresult = updatecheck.check(
File "/usr/local/munki/munkilib/updatecheck/core.py", line 82, in check
mainmanifestpath = manifestutils.get_primary_manifest(client_id)
File "/usr/local/munki/munkilib/updatecheck/manifestutils.py", line 175, in get_primary_manifest
manifest = get_manifest(clientidentifier, suppress_errors=True)
File "/usr/local/munki/munkilib/updatecheck/manifestutils.py", line 121, in get_manifest
dummy_value = fetch.munki_resource(
File "/usr/local/munki/munkilib/fetch.py", line 439, in munki_resource
return getResourceIfChangedAtomically(url,
File "/usr/local/munki/munkilib/fetch.py", line 394, in getResourceIfChangedAtomically
changed = getHTTPfileIfChangedAtomically(
File "/usr/local/munki/munkilib/fetch.py", line 526, in getHTTPfileIfChangedAtomically
header = get_url(url,
File "/usr/local/munki/munkilib/fetch.py", line 247, in get_url
options = middleware.process_request_options(options)
File "/usr/local/munki/middleware_gcs.py", line 115, in process_request_options
options["url"] = gcs_query_params_url(options["url"])
File "/usr/local/munki/middleware_gcs.py", line 108, in gcs_query_params_url
url = gen_signed_url(file_path)
File "/usr/local/munki/middleware_gcs.py", line 88, in gen_signed_url
key, client_id = read_keystore()
File "/usr/local/munki/middleware_gcs.py", line 76, in read_keystore
key, client_email = _read_cfpref_keystore()
File "/usr/local/munki/middleware_gcs.py", line 71, in _read_cfpref_keystore
key = load_privatekey(FILETYPE_PEM, key)
File "/usr/local/munki/Python.framework/Versions/3.8/lib/python3.8/site-packages/OpenSSL/crypto.py", line 2916, in load_privatekey
_raise_current_error()
File "/usr/local/munki/Python.framework/Versions/3.8/lib/python3.8/site-packages/OpenSSL/_util.py", line 57, in exception_from_error_queue
raise exception_type(errors)
OpenSSL.crypto.Error: [('PEM routines', 'get_name', 'no start line')]

On a side note, I agree the key is better formatted as Base64 in the profile, as otherwise you may run into issues with newlines. I removed these from my privkey just in case.

[patgmac]

@securitygeneration I'm getting the same error you are, were you able to figure anything out?

[securitygeneration]

@securitygeneration I'm getting the same error you are, were you able to figure anything out?

Nope 😔