WhoDidITalkTo: working ReactJS code!

You ever take a very long time to birth something small but ultimately, personally meaningful?

Me neither, but what I’m calling stage 1 of my ReactJS app is working to my liking.

WhoDidITalkTo is a personal work of love to help me remember all the wonderful encounters I have at Meetups and other such networking events.  It’s painful for me to keep forgetting the awesome conversations I’ve had with people, and have to confess I don’t remember someone who I very clearly made an impression on.  As someone with superhuman empathy, it’s crushing to see those hurt microexpressions cross their faces when they realize I’m no better than Leonard Shelby:

A little less dirty than him, usually

So I’m trying to remedy that, by giving myself a tool I can use from my phone to capture and review salient details from each new personal encounter I have at all the events I slut around to.

It’s prototype stage, and I have no dreams of monetizing this (so many startups have tried and failed to make this kind of “personal CRM lite” work and failed), and it’s a long ways from being fully functional in the field.  Still, I’m having fun seeing just how far I can stretch my rusty front end skills *and* treat this like a real Product Management project for myself.

If you’d like to peer inside my jumbled mind, this isn’t a bad place to see for yourself:

WhoDidITalkTo prototype v1

Notes to self: merging my fork with upstream

It’s supposed to be as natural as breathing, right?  See a neat repository on Github, decide you want to use the code and make some minor changes to it right?  So you fork the sucker, commit some change, maybe push a PR back to the original repo?

Then, you want to keep your repo around – I dunno, maybe it’s for vanity, or maybe you’re continuing to make changes or use the project (and maybe, just maybe, you’ll find yourself wanting to push another PR in the future?).  Or maybe messages like this just bother your OCD:


Eventually, most developers will run into a situation in which they wish to re-sync their forked version of a project with the updates that have been made in “upstream”.

Should be dead easy, yes?  People are doing this all the time, yes?  Well, crap.  If that’s the case, then I’m an idiot because I’d tried this a half-dozen times and never before arrived at the beautiful message “This branch is even with…”.  So I figured I’d write it out (talk to the duck), and in so doing stumble on the solution.

GitHub help is supposed to help, e.g. Syncing a fork.  Which depends on Configuring a remote for a fork, and which is followed by Pushing to a remote.

Which for a foreign repo named e.g. “hackers/hackit” means the following stream of commands (after I’ve Forked the repo in GitHub.com and git clone‘d the repo on my local machine):

git remote add upstream git@github.com:hackers/hackit.git
git fetch upstream
git checkout master
git merge upstream/master

That last command will often result in a bunch of conflicts, if you’ve made any changes, e.g.:

git merge upstream/master
Auto-merging package.json
CONFLICT (content): Merge conflict in package.json
Auto-merging README.md
Auto-merging .travis.yml
CONFLICT (content): Merge conflict in .travis.yml
Auto-merging .babelrc
Automatic merge failed; fix conflicts and then commit the result.

At this point I temporarily abandon the command line and dive into my favourite editor (Visual Studio Code with a handful of extensions) to resolve the conflicting files.

Once I’d merged changes from both sources (mine and upstream), then it was a simple matter of the usual commands:

git add .
git commit -m "merged changes from upstream"
git push

And the result is…


(No it wasn’t quite the “even” paradise, but I’ll take it.)


I somehow got myself into a state where I couldn’t get the normal commands to work.  For example, when I ran git push origin master, I get nowhere:

git push origin master
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

Or git push:

git push
ERROR: Permission to hackers/hackit.git denied to MikeTheCanuck.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

Then when I added upstream…:

git remote add upstream git@github.com:hackers/hackit.git

…and ran git remote -v…:

git remote -v
upstream git@github.com:hackers/hackit.git (fetch)
upstream git@github.com:hackers/hackit.git (push)

…it appears I no longer had a reference to origin. (No idea how that happened, but hopefully these notes will help me not go astray again.)  Adding back the reference to origin seemed the most likely solution, but I didn’t get the kind of results I wanted:

git remote add origin git@github.com:mikethecanuck/hackit.git
git remote -v
origin git@github.com:mikethecanuck/hackit.git (fetch)
origin git@github.com:mikethecanuck/hackit.git (push)
upstream git@github.com:hackers/hackit.git (fetch)
upstream git@github.com:hackers/hackit.git (push)
git push origin master
To github.com:mikethecanuck/hackit.git
 ! [rejected]        master -> master (fetch first)
error: failed to push some refs to 'git@github.com:mikethecanuck/hackit.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.


And when I pushed with no params, I went right back to the starting place:

git push
ERROR: Permission to hackers/hackit.git denied to MikeTheCanuck.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

(I finally rm -rf‘d my forked repo, cloned it again, and started over – that’s how I got to the first part of the article.)

Update my Contacts with Python: using pyobjc, Contacts.app & vCards, Swift or my own two hands?

I’m still on a mission to update my iCloud Contacts using PyiCloud to consolidate the data I’ve retrieved from LinkedIn.  Last time I convinced myself to add an update_contact() function to a fork of PyiCloud’s contacts module, and so far I haven’t had any nibbles on the issue I’ve filed in the PyiCloud project a couple of days ago.

I was looking further at the one possibly-working pattern in the PyiCloud project that appears to implement a write back to the iCloud APIs: the reminders module with its post() method.  What’s interesting to me is that in that method, the JSON submitted in the data parameter includes the key:value pair “etag”: None.  I gnashed my teeth over how to construct a valid etag in my last post, and this code implies to me (assuming it’s still valid and working against the Reminders API) that the etag value is optional (well, the key must be specified, but the complicated value may not be needed).

Knowing that this sounds too easy, I watched a new Reminder getting created through the icloud.com web client, and sure enough Chrome Dev Tools shows me that in the Request Payload, etag is set to null.  Which really tells me nothing now about the requirement for the Contacts API…

Arrested Development

Knowing that this was going to be a painful brick wall to climb, I decided to pair up with a python expert to look for ways to dig out from this deep, dark hole.  Lucky me, I have a good relationship with the instructor from my python class from late last year.  We talked about where I am stuck and what he’d recommend I do to try to break through this issue.

His thinking?  He immediately abandoned the notion of deciphering an undocumented API and went looking around the web for docs and alternatives.  Turns out there are a couple of options:

  1. Apple has in its SDKs a Contacts framework that supports Swift and Objective-C
  2. There are many implementations of Python & other languages that access the MacOS Contacts application (Contacts.app)

Contacts via Objective-C on MacOS

  • Contacts Framework is available in XCode
  • There appears to be a bidirectional bridge between Python and Objective-C
  • There is further a wrapper for the Contacts framework (which gets installed when you run pip install pyobjc)
  • But sadly, there is nothing even resembling a starter kit example script for instantiating and using the Contacts framework wrapper

Contacts via Contacts.app on MacOS

  • We found a decent-looking project (VObject) that purports to access VCard files, which is the underlying  data layout for import/export from Contacts.app
  • And another long-lived project (vcard) for validating VCards
  • This means I would have to manually import VCard file(s) into Contacts.app, and would still have to figure out how Contacts.app knows how to match/overwrite an imported Contact with an existing Contact (or I’ll always be backing up, deleting and importing)
  • HOWEVER, in exploring the content of the my Contacts.app and comparing to what I have in my iPhone Contacts, there’s definitely something extra going on here
    • I have at least one contact displayed in Contacts.app who is neither listed in my iPhone/iCloud contacts nor Google Contacts – given the well-formed LinkedIn data in the contact record, I’m guessing this is being implicitly included via Internet Accounts (the LinkedIn account configured here):
    • What would happen if I imported a vCard with the same UID (the iCloud UUID)?
    • What would happen if I imported a vCard that exists in both iCloud and LinkedIn – would the iCloud (U)UID correctly match and merge the vCard to the right contact, or would we get a duplicate?
  • Here at least I see others acknowledge that it’s possible to create non-standard types for ADR, TEL (and presumably email and URL types, if they’re different).
  • Watch out: if you have any non-ASCII characters in your Address Book, exporting will generate the output as UTF-16.
  • Watch out: here’s a VObject gotcha.

Crazy Talk: Swift?

  • I *could* go learn enough Swift to interface with the JSON data I construct in Python
  • There’s certainly a plethora of articles (iOS-focused) and tutorials to help folks use the Contacts framework via Swift – which all seem to assume you want to build a UI app (not just a script) – I guess I understand the bias, but boy do I feel left out just wanting to create a one-time-use script to make sure I don’t fat-finger something and lose precious data (my wetware memory is lossy enough as it is)

Conclusion: Park it for now

What started out as a finite-looking piece of work to pull LinkedIn data into my current contacts of record, turned into a never-ending series of questions, murky code pathfinding  and band-aiding multiple technologies together to do something that should ostensibly be fairly straightforward.

Given that the best options I have at this point are (a) reverse-engineer an undocumented Apple API, (b) try to leverage an Objective-C bridge that *no one* else has tried to use for this new Contacts framework, or (c) decipher how Contacts.app interacts in the presence of vCards and all the interlocking contacts services (iCloud, Google, LinkedIn, Facebook)…I’m going to step away from this for a bit, let my brain tease apart what I’m willing to do for fun and how much more effort I’m willing to put in for fun/”the community”, and whether I’ve crossed The Line for a one-time effort and should just manually enter the data myself.

Notes to self: installing Vagrant via homebrew

It’s as easy as

brew install vagrant

Ha! Nice try there, buddy:

Error: No available formula with the name "vagrant" 
==> Searching for similarly named formulae...
Error: No similarly named formulae found.
==> Searching taps...
These formulae were found in taps:
homebrew/completions/vagrant-completion  Caskroom/cask/vagrant-manager
Caskroom/cask/vagrant-bar                Caskroom/cask/vagrant
To install one of them, run (for example):
  brew install homebrew/completions/vagrant-completion

According to this discussion, it’s not allowed by homebrew but the homebrew-cask project enables users to install vagrant.  (So *that’s* why I couldn’t get Chrome, Dropbox, 1Password, VLC and other apps installed view homebrew – there’s some rule or constraint in homebrew that only enables non-GUI apps.)

So let’s install the Cask tools – this article makes me feel silly even – you just have to know to use the “cask” keyword, as in:

brew cask install vagrant

Thus do I get vagrant v1.9.1 in one (slightly unexpected) command line.  (And don’t forget virtualbox and vagrant-manager!)

Which is super-cool, because many package managers end up with non-current builds of the tools in their catalogs.

And as a bonus, they mention a bunch of quicklook plugins that I never even thought to go looking for – markdown, syntax-highlighted code,  JSON, CSV and more!

Parsing PDFs using Python

I’m part of a project that has a need to import tabular data into a structured database, from PDF files that are based on digital or analog inputs.  [Digital input = PDF generated from computer applications; analog input = PDF generated from scanned paper documents.]

These are the preliminary research notes I made for myself a while ago that I am now publishing for reference by other project members.  These are neither conclusive nor comprehensive, but they are directionally relevant.

I.E. The amount of work it takes code to parse structured data from analog input PDFs is a significant hurdle, not to be underestimated (this blog post was the single most awe-inspiring find I made).  The strongest possible recommendation based on this research is GET AS MUCH OF THE DATA FROM DIGITAL SOURCES AS YOU CAN.


Evaluation of Packages

Possible issues

  • Encryption of the file
  • Compression of the file
  • Vector images, charts, graphs, other image formats
  • Form XObjects
  • Text contained in figures
  • Does text always appear in the same place on the page, or different every page/document?

PDF examples I tried parsing, to evaluate the packages

  • IRS 1040A
  • 2015-16-prelim-doc-web.pdf (Bellingham city budget)
    • Tabular data begins on page 30 (labelled Page 28)
    • PyPDF2 Parsing result: None of the tabular data is exported
    • SCARY: some financial tables are split across two pages
  • 2016-budget-highlights.pdf (Seattle city budget summary)
    • Tabular data begins on page 15-16 (labelled 15-16)
    • PyPDF2 Parsing result: this data parses out
  • FY2017 Proposed Budget-Lowell-MA (Lowell)
    • Financial tabular data starts at page 95-104, then 129-130, 138-139
    • More interesting are the small breakouts on subsequent pages e.g. 149, 151, 152, 162; 193, 195, 197
    • PyPDF2 Parsing result: all data I sampled appears to parse out

Experiment ideas

  • Build an example PDF for myself with XLS tables, and then see what comes out when the contents are parsed using one of these libraries
  • Build a script that spits out useful metadata about the document: which app/library generated it (e.g. Producer, Creator), size, # of pages
  • Build another script to verify there’s a non-trivial amount of ASCII/Unicode text in the document (I.e. so we confirm it doesn’t have to be OCR’d)

Experiments tried

Scaling the Cliffs of Insanity (aka using Ansible from a Windows controller)

Dread Pirate Roberts and Princess Buttercup, out for a morning stroll

This post is just record-keeping for me to remember why I abandoned Ansible on Windows a few months back.

Seriously, here be dragons

Here’s what I did to enable me to use Ansible for automation

  • Since I’m using a Windows box as my primary desktop for now, I wanted to see if I could avoid running a Linux VM just to manage other *NIX boxes
  • I thought I should be able to get away with using the Git for Windows environment rather than the full Cygwin environment that Jeff Geerling documents here (spoiler alert: I couldn’t, but it’s instructive to document how far I got and where it broke down)

Install and troubleshoot Ansible on Git for Windows

  • I followed installation instructions from step (3) (including using those legacy versions of PyYAML and Jinja2)
  • I skipped step (6) since I already had SSH keys
  • I ran ansible –version, and saw errors like:
     $ ansible --version
     Traceback (most recent call last):
       File "C:/Users/Mike/code/ansible/bin/ansible", line 46, in <module>
        from ansible.utils.display import Display
       File "C:\Users\Mike\code\ansible\lib\ansible\utils\display.py", line 21, in <module>
        import fcntl
     ImportError: No module named 'fcntl'
  • Tried resolving that dependency via http://stackoverflow.com/questions/1422368/fcntl-substitute-on-windows
  • This just led to the following (which seemed a hint that I might encounter many such issues, not worth it):
     $ ansible --version
     Traceback (most recent call last):
       File "C:/Users/Mike/code/ansible/bin/ansible", line 46, in <module>
        from ansible.utils.display import Display
       File "C:\Users\Mike\code\ansible\lib\ansible\utils\display.py", line 33, in <module>
        from termios import TIOCGWINSZ
     ImportError: No module named 'termios'
  • Gave in, uninstalled Python 3.5.1 and jumped back to latest Python 2.x (2.7.1)
  • Still couldn’t get past that error, so I gave up and went to Cygwin

Install and troubleshoot Ansible on Cygwin64

Then I found this article, followed a combo of the top answer:

  • apt-cyg remove python-cryptography
  • git clone –depth 1 git://github.com/ansible/ansible
  • apt-cyg install git python-{jinja2,six,yaml}
  • wget rawgit.com/transcode-open/apt-cyg/master/apt-cyg && install apt-cyg /bin
  • That results in this output:
     $ ansible --version
     ansible 2.2.0 (devel 37737ca6c1) last updated 2016/05/11 14:27:18 (GMT -700)
       lib/ansible/modules/core:  not found - use git submodule update --init lib/ansible/modules/core
       lib/ansible/modules/extras:  not found - use git submodule update --init lib/ansible/modules/extras
       config file =
       configured module search path = ['/opt/ansible/library']
  • Then running this command from within the local ansible repo (e.g. from /opt/ansible) gets the basic modules you’ll need to get started (e.g. the “ping” module for testing)
    git submodule update –init –recursive
  • Then I decided to test ansible’s ability to connect to the target host based on the command recommended in Ansible’s installation docs:
     $ ansible all -m ping --ask-pass
     SSH password: | FAILED! => {
        "failed": true,
        "msg": "to use the 'ssh' connection type with passwords, you must install the sshpass program"
  • OK, let’s get sshpass installed on my control device [Windows box] so that we can quickly bootstrap the SSH keys to the target…
  • Since sshpass isn’t an available package in Cygwin package directory, the only way to get sshpass is to install it from source
    •  NOTE: I discovered this the hard way, after trying every trick I could think of to make all this work without having to deal with a C compiler
  • Found this article for OS X users and followed it in my Windows environment (NOTE: I had to install the “make” and “gcccore” packages from Cygwin setup – and then having to re-run apt-cyg remove python-cryptography from the Cygwin terminal again because it gets automatically reinstalled by Cygwin setup)
  • This time when running the command I get a different error – which means I must’ve got sshpass stuffed in the right location:
     $ ansible all -m ping --ask-pass
     SSH password: | FAILED! => {
        "failed": true,
        "msg": "Using a SSH password instead of a key is not possible because Host Key checking is enabled and sshpass does not support this.  Please add this host's fingerprint to your known_hosts file to manage this host."
  • I’m guessing this means I need to obtain the target host’s SSH public key (and not that the target host refused to connect because it didn’t trust the control host).  So then I had to harvest the target host’s fingerprint
     $ ssh -o StrictHostKeyChecking=ask -l mike
     The authenticity of host ' (' can't be established.
     ECDSA key fingerprint is SHA256:RqBcq8aCAgohFeiTlPeYd8hLDfTz1A25ZPlzyxlDrqI.
     Are you sure you want to continue connecting (yes/no)? yes
     Warning: Permanently added '' (ECDSA) to the list of known hosts.
     mike@'s password:
  • This time around when running the ping command I saw this error:
     $ ansible all -m ping --ask-pass
     SSH password: | UNREACHABLE! => {
        "changed": false,
        "msg": "Authentication failure.",
        "unreachable": true
  • I finally got the bright idea to poke around the target’s log files and see if there were any clues – sure enough, /var/log/auth.log made it clear enough to me:
     Invalid user Mike from
     input_userauth_request: invalid user Mike [preauth]
      pam_unix(sshd:auth): check pass; user unknown
      pam_unix(sshd:auth): authentication failure...
      Failed password for invalid user Mike from port 58665 ssh2
      Connection closed by [preauth]
  • (Boy do I hate *NIX case-sensitivity at times like this – my Windows username on the control device is “Mike” and the Linux username on the target is “mike”)
  • Tried this same command but adding the root user (ansible all -m ping –user root –ask-pass) but kept getting the same error back, and saw /var/log/auth.log reported “Failed password for root from”)
  • Re-ran su root in bash on the target just to make sure I wasn’t forgetting the password (I wasn’t)
  • Tried a basic ssh connection with that same password (to eliminate other variables):
     $ ssh root@
     root@'s password:
     Permission denied, please try again.
     root@'s password:
     Permission denied, please try again.
     root@'s password:
     Permission denied (publickey,password).
  • Tried the same thing with my “mike” user, success:
     $ ssh mike@
     mike@'s password:
     The programs included with the Debian GNU/Linux system are free software;
     the exact distribution terms for each program are described in the
     individual files in /usr/share/doc/*/copyright.
     Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
     permitted by applicable law.
     Last login: Sat May 14 16:10:05 2016 from
  • One more try with the ansible/SSH/password approach:
     $ ansible all -m ping --user mike --ask-pass
     SSH password: | UNREACHABLE! => {
        "changed": false,
        "msg": "Failed to connect to the host via ssh.",
        "unreachable": true
  • So close!!  Search led to this possibility, so I re-ran with -vvvv param to get this:
     $ ansible all -m ping --user mike --ask-pass -vvvv
     No config file found; using defaults
     SSH password:
     Loaded callback minimal of type stdout, v2.0
     <> SSH: EXEC sshpass -d44 ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o User=mike -o ConnectTimeout=10 -o ControlPath=/home/Mike/.ansible/cp/ansible-ssh-%h-%p-%r '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1463273031.8-128175022355609 `" && echo "` echo $HOME/.ansible/tmp/ansible-tmp-1463273031.8-128175022355609 `" )'"'"''
     <> PUT /tmp/tmp948_ec TO /home/mike/.ansible/tmp/ansible-tmp-1463273031.8-128175022355609/ping
     <> SSH: EXEC sshpass -d44 sftp -b - -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o User=mike -o ConnectTimeout=10 -o ControlPath=/home/Mike/.ansible/cp/ansible-ssh-%h-%p-%r '[]' | UNREACHABLE! => {
        "changed": false,
        "msg": "SSH Error: data could not be sent to the remote host. Make sure this host can be reached over ssh",
        "unreachable": true
  • I can definitely see the ansible-tmp-1463273031.8-128175022355609 file under ~/.ansible/tmp on the target system, so Ansible is getting authenticated and can run the initial shell commands
  • But I’m not seeing the /ping file under that directory, and I’m wondering if there’s something preventing sftp from connecting to the target host (since that’s the final command being run just before I get back the error). The “sftp” program is available on the controller though.
  • Digging around in Wireshark, I can see the SSHv2 traffic between controller and target, but after the initial key exchange I see exactly 3 encrypted packets sent from the controller (and encrypted responses from the target), and then no further communication between the two thereafter.  The only other activity I see on either system that’s unexplained is the target device ARP’ing for the local router three times, once every second, after the SSH traffic dies off
  • After the first two initial successes in getting the tmp files pushed via ssh, I’ve since only had failures “Failed to connect to the host via ssh.”  Using ProcExp.exe to verify that there is actual network traffic being sent to the target’s IP, and using Wireshark to get some idea what’s getting through and what’s not (but Wireshark is acting up and no longer showing me traffic from controller to target, only target to controller responses, so it’s getting a little nuts at this point)
  • I’ve added “PTR” records to the hosts files on both the controller and the target to resolve the IP address for each other to a defined name, but I’m still getting “failed to connect…” (even though I can confirm that the tools are using the newly-registered name, since I even tried substituting the falsified name in the ansible_hosts file for the previously-used IP address)
  • I tried the advice from here to switch to SCP if SFTP might not be working, but that didn’t help (so I emptied out the .ansible.cfg file again)
  • I don’t know where to look for log files to see exactly what errors are occurring locally, so I’m pretty much stumped at this point
  • !!!!!! I pushed my ssh key to the target host, and the very next run of ansible all -m ping succeeded!!!! 😦
  • CONCLUSION: ssh-pass doesn’t seem compatible with the Windows setup I’ve been using all weekend
  • EPILOGUE: “Failed to connect…” error is back again when running ansible from Windows – I can see successful auth in the target’s /var/log/auth.log, but even -m ping fails (e.g. ansible all -m ping -u root)

Other References



Simplifying Vagrant-based testing: unsolved (I’m just calling it out to the universe)

I’m doing some pretty mind-numbing testing using Vagrant (yes, on Windows 10 – I like the challenge, apparently!), to make sure that I’m getting the results from changes I’m making to Ansible scripts.  Currently I’m testing the implementation of Ansible Vault, which means at each step of testing I:

  1. Vagrant destroy whatever box I just worked on
    • Which half the time means Vagrant and Virtualbox get out of sync, and I need to delete files and just vagrant init)
  2. Vagrant up
    • If I just init’d a new box, then I have to go into the Vagrantfile to uncomment then edit the config.vm.synced_folder setting, so that it removes the rsync dependency (setting it to config.vm.synced_folder “.”, “/vagrant”, disabled:true) – otherwise, vagrant up halts when it can’t find an rsync executable
  3. Mount the VM in Virtualbox Manager – Machine, Add…, find the .vbox file), then  launch the VM from VBox Mgr, login as vagrant, and edit the /etc/ssh/sshd_config file to set all instances of PasswordAuthentication to “yes”
  4. Reboot the VM
  5. Vagrant up
  6. Run ssh-keygen -f “/home/mike/.ssh/known_hosts” -R []:2222 to clear out the previously-trusted host SSH key
  7. Run ssh-copy-id vagrant@ -p 2222 to add my user’s SSH public key to the remote system (to enable Ansible to run over SSH)

I haven’t had time yet to start researching how to troubleshoot/automate each of these steps, but which I’ll eventually have to conquer so that I’m not re-learning the manual steps every time I return to volunteering a little spare time to this infrastructure project.

Why doesn’t chmod under Bash on Ubuntu on Windows 10 actually “take”?

I’m continuing to beat my head against a wall, attempting to test a very simple configuration change to an Ansible playbook I wrote, so that I can verify if my understanding of the use of Ansible vault is correct.

The latest problem?  Unix permissions.

Now that I’ve got SSH communications working between by Bash shell (Ubuntu on Windows 10, aka WSL), I’ve implemented changes to the playbook’s files including creating a .vault_pass.txt file under the Bash shell, and encrypting a vault.yml file using the password contained in the .vault_pass.txt.

When I run ansible-playbook role.yml –vault-password-file .vault_pass.txt, it complains of the following:

mike@MIKE-WIN10-SSD:~/code/ansible-role-unattended-upgrades$ ansible-playbook role.yml --vault-password-file .vault_pass.txt
ERROR! Problem running vault password script / m n t / c / U s e r s / M i k e / c o d e / C o p y - a n s i b l e - r o l e - u n a t t e n d e d - u p g r a d e s / . v a u l t _ p a s s . t x t ([Errno 8] Exec format error). If this is not a script, remove the executable bit from the file.

No problem, I’ve got this.  Just gotta run chmod 600 (or similarly, to remove the execute bit for my user) on the .vault_pass.txt file.  [For comparison, I just tried this on the same configuration under Ubuntu – which is having a different blocking issue at present, but not related to file permissions – and the command took immedateily.]  Hah, you should be so lucky:

mike@MIKE-WIN10-SSD:~/code/ansible-role-unattended-upgrades$ ls -la .vault_pass.txt
-rwxrwxrwx 1 root root 25 Sep 26 18:38 .vault_pass.txt
mike@MIKE-WIN10-SSD:~/code/ansible-role-unattended-upgrades$ chmod 600 .vault_pass.txt
mike@MIKE-WIN10-SSD:~/code/ansible-role-unattended-upgrades$ ls -la .vault_pass.txt
-rwxrwxrwx 1 root root 25 Sep 26 18:38 .vault_pass.txt
mike@MIKE-WIN10-SSD:~/code/ansible-role-unattended-upgrades$ whoami
mike@MIKE-WIN10-SSD:~/code/ansible-role-unattended-upgrades$ sudo chmod 600 .vault_pass.txt
[sudo] password for mike:
mike@MIKE-WIN10-SSD:~/code/ansible-role-unattended-upgrades$ ls -la .vault_pass.txt
-rwxrwxrwx 1 root root 25 Sep 26 18:38 .vault_pass.txt

Yes, I get that the file is owned by root, and I’m running as mike – so why doesn’t it make a difference when I run sudo chmod?  Is this a problem with files owned by root?  Is this a problem with chmod?  Is this a problem with WSL/Bash?

Lightbulb moment

I went hunting for such issues in the Microsoft repo for the Bash On Windows project, and found this issue & comment:


So I figured I re-examine the situation.  All my files under the ~/code folder are owned by root – even . and .., which is odd…

mike@MIKE-WIN10-SSD:~/code$ ls -la
total 68
drwxrwxrwx 2 root root 0 Sep 26 10:51 .
drwxrwxrwx 2 root root 0 Aug 16 17:00 ..
drwxrwxrwx 2 root root 0 Aug 16 16:28 ansible-role-unattended-upgrades

Then I looked at my home folder and – d’oh!

mike@MIKE-WIN10-SSD:~$ ls -la
total 24
drwxr-xr-x 2 mike mike 0 Sep 26 18:37 .
drwxr-xr-x 2 root root 0 Dec 31 1969 ..
-rw------- 1 mike mike 2452 Aug 16 22:48 .bash_history
-rw-r--r-- 1 mike mike 220 Aug 5 10:06 .bash_logout
-rw-r--r-- 1 mike mike 3637 Aug 5 10:06 .bashrc
lrwxrwxrwx 1 mike mike 22 Aug 16 12:58 code -> /mnt/c/Users/Mike/code


Now I remember: when I first sat down with this Bash On Ubuntu on Windows setup, I figured I’d save myself some trouble by using the exact same files in all my local repos – why bother duplicating the repos between Windows and Bash on Ubuntu?  So I symlinked a mount of the /code folder from my Windows user profile…and left myself a nice little landmine, it seems.

Rather than struggle with cacls.exe and try to find some magic combination that results in non-executable permissions on that file through the WSL translation layer (if at all), I just cloned the repo to a different folder (local to the Bash/Ubuntu/Win10 environment) and retried, with trivial success.

Simple troubleshooting the usual SSH error from Ansible

After the struggles I’ve had over the last couple of days, it’s strangely reassuring to stumble on a problem I’ve actually *seen* before, and recently.  Firsthand even.

I’ve fresh-built a Debian 8.5.2 VM in Virtualbox via Vagrant.  Then I setup an Ansible inventory file to point to the box’s current vagrant ssh-config settings.  Then fired off the tried-and-true ansible connectivity test, ansible all -u vagrant -m ping.  Here’s the response: | UNREACHABLE! => {
    "changed": false,
    "msg": "ERROR!  SSH encountered an unknown error during the connection.  We recommend you re-run the command using -vvvv, which will enable SSH debugging output to help diagnose the issue",
    "unreachable": true

Running the same command with -vvvv parameter results in a garbage heap of unformatted/concatenated debug screed, which ends with:

debug1: No more authentication methods to try.\r\nPermission denied (publickey,password).\r\n"

Simple Solution

As I’ve documented to myself already elsewhere, I need to run the following two commands:

ssh-keygen -t rsa

ssh-copy-id -p 2222 vagrant@

Bingo!  Nice to get an easy win.

Troubleshooting SSH blocker in the Debian/jessie64 Vagrant box

After getting Vagrant and Virtualbox to play nice together, I turned my attention back to testing my Ansible Vault configuration ideas on a Debian8 VM.

Because I’d been having continued problems connecting to the damned box, I init’d a new VM based on debian/jessie64.  Once again, however, I noticed two issues:

  1. Vagrant is no longer registering new VMs in the VirtualBox Manager (when did Vagrant stop doing this?)
  2. I’m unable to copy SSH keys to freshly-booted Debian VM – ssh-copy-id results in this response:
    mike@MIKE-WIN10-SSD:~/$ ssh-copy-id vagrant@ -p 2200
    /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
    /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
    Permission denied (publickey).

This article purports to have the answer to this, but even after fixing and re-checking the /etc/ssh/sshd_config multiple times, I’m still getting the same “Permission denied (publickey)” response.

After a couple of hours of tail-chasing, I finally remembered the debug flags in the ssh command (-v and -vvv).  That results in this output:

mike@MIKE-WIN10-SSD:~/code/Copy-ansible-role-unattended-upgrades$ ssh -p 2200 vagrant@ -v
OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to [] port 2200.
debug1: Connection established.
debug1: identity file /home/mike/.ssh/id_rsa type 1
debug1: identity file /home/mike/.ssh/id_rsa-cert type -1
debug1: identity file /home/mike/.ssh/id_dsa type -1
debug1: identity file /home/mike/.ssh/id_dsa-cert type -1
debug1: identity file /home/mike/.ssh/id_ecdsa type -1
debug1: identity file /home/mike/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/mike/.ssh/id_ed25519 type -1
mike@MIKE-WIN10-SSD:~/code/Copy-ansible-role-unattended-upgrades$ ssh -p 2200 vagrant@ -v
OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to [] port 2200.
debug1: Connection established.
debug1: identity file /home/mike/.ssh/id_rsa type 1
debug1: identity file /home/mike/.ssh/id_rsa-cert type -1
debug1: identity file /home/mike/.ssh/id_dsa type -1
debug1: identity file /home/mike/.ssh/id_dsa-cert type -1
debug1: identity file /home/mike/.ssh/id_ecdsa type -1
debug1: identity file /home/mike/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/mike/.ssh/id_ed25519 type -1
debug1: identity file /home/mike/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.8
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.7p1 Debian-5+deb8u3
debug1: match: OpenSSH_6.7p1 Debian-5+deb8u3 pat OpenSSH* compat 0x04000000
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-sha1-etm@openssh.com none
debug1: kex: client->server aes128-ctr hmac-sha1-etm@openssh.com none
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ECDSA b0:b7:27:f4:0a:91:a4:37:8c:ce:35:a3:e3:fe:db:2d
debug1: Host '[]:2200' is known and matches the ECDSA host key.
debug1: Found key in /home/mike/.ssh/known_hosts:4
debug1: ssh_ecdsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/mike/.ssh/id_rsa
debug1: Authentications that can continue: publickey
debug1: Trying private key: /home/mike/.ssh/id_dsa
debug1: Trying private key: /home/mike/.ssh/id_ecdsa
debug1: Trying private key: /home/mike/.ssh/id_ed25519
debug1: No more authentication methods to try.
Permission denied (publickey).

I tried regenerating keys, but that still ends with the same “Permission denied (publickey)” message.

Is it possible that the Debian box’s sshd isn’t accepting RSA keys for SSH auth?  If that were true, wouldn’t the sshd_config include “RSAAuthentication no” instead of the “RSAAuthentication yes” I’m seeing?

It’s odd – when I attempt to ssh directly, I’m getting this kind of output – this implies that the remote sshd is attempting to accept password for authentication, even though it’s acting like I haven’t typed in the correct password (I am):

mike@MIKE-WIN10-SSD:~/code/Copy-ansible-role-unattended-upgrades$ ssh vagrant@ 2200
vagrant@'s password:
Permission denied, please try again.
vagrant@'s password:
Permission denied, please try again.
vagrant@'s password:
Received disconnect from 14:

Time for a reset.

Solution (?)

Ripped out every VM on my system.  Re-inited.  Edited Vagrantfile.  Tried/failed.  Mounted the machine in the VirtualBox Manager app (because, Vagrant’s still not registering the machine with the Manager UI). Launched the Debian box interactively from VBox Mgr.

Edited the /etc/ssh/sshd_config file to change the PasswordAuthentication setting TWICE.


Something, somewhere, is inserting two entries (one commented out, the other uncommented at the very end of the file) that are both set to “no”.

What.  The.  Heck.

Set them *both* to yes (left them both uncommented, just for show) and rebooted the box.


ssh-copy-id is easily able to authenticate with the vagrant password *and* copy the current RSA public key to the appropriate file:

mike@MIKE-WIN10-SSD:~/$ ssh-copy-id vagrant@ -p 2222
The authenticity of host '[]:2222 ([]:2222)' can't be established.
ECDSA key fingerprint is b0:b7:27:f4:0a:91:a4:37:8c:ce:35:a3:e3:fe:db:2d.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
Permission denied (publickey).
mike@MIKE-WIN10-SSD:~/code/Copy-ansible-role-unattended-upgrades$ ssh-copy-id vagrant@ -p 2222
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
vagrant@'s password:

Number of key(s) added: 1

Now try logging into the machine, with: "ssh -p '2222' 'vagrant@'"
and check to make sure that only the key(s) you wanted were added.

Boy, *that’s* going to be fun to remember to do every time I destroy and recreate this VM.

[Oh, and keep this article in your back pocket in case you run into a different SSH troubleshooting issue: http://askubuntu.com/questions/311558/ssh-permission-denied-publickey]