These are notes I kept for myself while struggling to find a way to use Vagrant and Virtualbox (among other tools like git, ansible and ssh) to test and debug virtual server builds without having to dual-boot into a Linux box.
Note to DavidP: you’ll see throughout the rsync errors that made this whole thing not *quite* a functional vagrant environment.
Obligatory: “this was of course a foolish mission that many loudly warned me against”.
Obligatory: “I thought I was smarter than most, and could probably decipher the grotty issues” arrogance.
Futile: “I hope these notes help someone else somewhere down the line with a similar issue”.
Important: my work was liberally stealing great ideas from Stefan van Essen:
Basic Setup
Details of what I did to make sure I can recreate my environment later:
- Installed Cygwin64 with tools like bash, curl, gcc-core, make, nano, openssh, openssl, python, wget (and keep uninstalling python-cryptography because it interferes with my almost-working “ansible-under-cygwin” setup)
- Installed apt-cyg to make cygwin package management easier
- Installed VirtualBox for Windows with default paths (leaving the VMs in %username%/VirtualBox VMs)
- Installed Vagrant for Windows with no changes
- Created a symlink in the cygwin home directory to the VMs for ease of access (ln -s “/cygdrive/c/Users/Mike/VirtualBox VMs” ~/VMs)
First sign of trouble, selection of build target
- Problem: “vagrant up” runs into a couple of problems consistently, which might have something to do with me instantiating the vagrant box under Windows (where there wasn’t an available rsync program to enable Vagrant to do whatever it needs when first “up”ing a new Vagrant box)
- There seemed to be an SSH issue:
==> default: Waiting for machine to boot. This may take a few minutes... default: SSH address: 127.0.0.1:2222 default: SSH username: vagrant default: SSH auth method: private key default: Warning: Remote connection disconnect. Retrying... default: Warning: Remote connection disconnect. Retrying... ==> default: Machine booted and ready!
- Then appeared a “hangover” rsync issue:
==> default: Rsyncing folder: /cygdrive/c/Users/Mike/VirtualBox VMs/BaseDebian/ => /vagrant There was an error when attempting to rsync a synced folder. Please inspect the error message below for more info. Host path: /cygdrive/c/Users/Mike/VirtualBox VMs/BaseDebian/ Guest path: /vagrant Command: rsync --verbose --archive --delete -z --copy-links --chmod=ugo=rwX --no-perms --no-owner --no-group --rsync-path sudo rsync -e ssh -p 2222 -o ControlMaster=auto -o ControlPath=C:/cygwin64/tmp/ssh.570 -o ControlPersist=10m -o StrictHostKeyChecking=no -o IdentitiesOnly=true -o UserKnownHostsFile=/dev/null -i 'C:/Users/Mike/VirtualBox VMs/BaseDebian/.vagrant/machines/default/virtualbox/private_key' --exclude .vagrant/ /cygdrive/c/Users/Mike/VirtualBox VMs/BaseDebian/ vagrant@127.0.0.1:/vagrant Error: Warning: Permanently added '[127.0.0.1]:2222' (ECDSA) to the list of known hosts. mm_receive_fd: no message header process_mux_new_session: failed to receive fd 0 from slave mux_client_request_session: read from master failed: Connection reset by peer Failed to connect to new control master rsync: connection unexpectedly closed (0 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.2]
- I figured rather than treat this box as a pet, I’d just destroy it and try over again [good practice for breaking my old habits of trying to root cause catastrophic, but sometimes irreproducible, issues]
- But no – these issues reappear for a freshly-init’d instance of the same vagrant box (debian/jessie64)
- However, the next vagrant box I tried (hashicorp/precise64) (where I’ll put my dev tools) seemed close enough to fine, no red errors:
==> default: Machine booted and ready! ==> default: Checking for guest additions in VM... default: The guest additions on this VM do not match the installed version of default: VirtualBox! In most cases this is fine, but in rare cases it can default: prevent things such as shared folders from working properly. If you see default: shared folder errors, please make sure the guest additions within the default: virtual machine match the version of VirtualBox you have installed on default: your host and reload your VM. default: default: Guest Additions Version: 4.2.0 default: VirtualBox Version: 5.0 ==> default: Mounting shared folders... default: /vagrant => C:/Users/Mike/VirtualBox VMs/UbuntuDev
- I also tried with (box-cutter/ubuntu1404-desktop) – worked great!
==> default: Machine booted and ready! ==> default: Checking for guest additions in VM... ==> default: Mounting shared folders... default: /vagrant => C:/Users/Mike/VirtualBox VMs/Ubuntu-box-cutter
- Conclusion: using the box-cutter/ubuntu1404 box for my dev work
- There seemed to be an SSH issue:
Configuration woes
- Installed git (with suggested additions)
- Installed ansible
- Generated ssh keys, ssh-copy-id copied to mike@192.168.1.14 (debian1sttry), then ran “cp ~/.ssh/authorized_keys /root/.ssh” as root
- Next I wanted to share a single Code folder between Windows and the Ubuntu VM:
- 1st idea: create a symlink in the shared “vagrant” folder (C:\Users\Mike\VirtualBox VMs\Ubuntu-box-cutter) that would be addressable from inside the Ubuntu VM:
[from Cygwin on host] ln -s /cygdrive/c/Users/Mike/Code Code
[from Ubuntu VM]
vagrant@vagrant:~$ ls /vagrant/Code
/vagrant/Code
vagrant@vagrant:~$ cd /vagrant/Code
bash: cd: /vagrant/Code: Not a directory - 2nd idea: try a native-Windows symlink, not a Cygwin version
[from CMD on host]
C:\Users\Mike\VirtualBox VMs\Ubuntu-box-cutter>mklink /D Code C:\Users\Mike\Code
symbolic link created for Code <<===>> C:\Users\Mike\Code
[from Ubuntu VM]
vagrant@vagrant:/vagrant$ cd /vagrant/Code
bash: cd: /vagrant/Code: Not a directory - 3rd idea: create a new synced folder via vagrant
- 1st idea: create a symlink in the shared “vagrant” folder (C:\Users\Mike\VirtualBox VMs\Ubuntu-box-cutter) that would be addressable from inside the Ubuntu VM:
- Meanwhile, I started out a new repo (https://github.com/MikeTheCanuck/jQuery-infra-update) to contain the suggested Ansible code (using the Git Bash shell from Git for Windows):
-
mkdir code cd code touch 50unattended-upgrades touch 02periodic git add 50unattended-upgrades git add 02periodic
- [then tried git commit but got prompted for identity]
-
git config --global user.email "mikethecanuck@gmail.com" git config --global user.name "Mike Lonergan" git commit -m "First commit" git push origin master
- [fun: Windows Credential manager prompted for my Github creds]
-
- Now to follow the canonical git-flow http://nvie.com/posts/a-successful-git-branching-model/:
-
git checkout -b develop git checkout -b default-file-contents develop git commit -m "populated default content" code/02periodic code/50unattended-upgrades git push
- [I didn’t think it worked – I didn’t see any changes on the upstream repo – so I tried over]
-
git reset git reset --hard git commit -m "Added default config file contents" -a git push
- [which resulted in the warning “fatal: The current branch default-file-contents has no upstream branch. To push the current branch and set the remote as upstream, use…(command)”, so I did]
-
git push --set-upstream origin default-file-contents
-
- Result: I have two commits to the branch on my Git repo
Ansible testing environment setup
- 1st: setup my inventory in /etc/ansible/hosts
- 2nd: test connectivity “ansible dev -m ping -u root” [cause my target doesn’t have a vagrant account – yeah yeah, I’ll get to building a target using vagrant soon]
- 3rd: move the files over for testing:
ansible dev -u root -m copy -a "backup=yes src=~/code/jQuery-infra-update/code/50unattended-upgrades dest=/etc/apt/apt.conf.d/" ansible dev -u root -m copy -a "backup=yes src=~/code/jQuery-infra-update/code/02periodic dest=/etc/apt/apt.conf.d/"
Testing vagrant on Debian clones
- I installed a base Debian box and ignored this error during “vagrant up” that I hoped wouldn’t be a problem:
==> default: Checking for guest additions in VM... default: No guest additions were detected on the base box for this VM! Guest default: additions are required for forwarded ports, shared folders, host only default: networking, and more. If SSH fails on this machine, please install default: the guest additions and repackage the box to continue. default: default: This is not an error message; everything may continue to work properly, default: in which case you may ignore this message. ==> default: Rsyncing folder: /cygdrive/c/Users/Mike/VirtualBox VMs/BaseDebian/ => /vagrant There was an error when attempting to rsync a synced folder Please inspect the error message below for more info. Host path: /cygdrive/c/Users/Mike/VirtualBox VMs/BaseDebian/ Guest path: /vagrant Command: rsync --verbose --archive --delete -z --copy-links --chmod=ugo=rwX --no-perms --no-owner --no-group --rsync-path sudo rsync -e ssh -p 2222 -o ControlMaster=auto -o ControlPath=C:/cygwin64/tmp/ssh.275 -o ControlPersist=10m -o StrictHostKeyChecking=no -o IdentitiesOnly=true -o UserKnownHostsFile=/dev/null -i 'C:/Users/Mike/VirtualBox VMs/BaseDebian/.vagrant/machines/default/virtualbox/private_key' --exclude .vagrant/ /cygdrive/c/Users/Mike/VirtualBox VMs/BaseDebian/ vagrant@127.0.0.1:/vagrant Error: Warning: Permanently added '[127.0.0.1]:2222' (ECDSA) to the list of known hosts. mm_receive_fd: no message header process_mux_new_session: failed to receive fd 0 from slave mm_send_fd: sendmsg(2): Broken pipe mux_client_request_session: send fds failed rsync: connection unexpectedly closed (0 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.2]
- Now I’m working from the BaseDebian box I’d setup, and tried to clone a couple of boxes from it, but found that they complained that the VBox Additions weren’t installed and wouldn’t allow me to proceed:
==> default: Checking for guest additions in VM... default: No guest additions were detected on the base box for this VM! Guest default: additions are required for forwarded ports, shared folders, host only default: networking, and more. If SSH fails on this machine, please install default: the guest additions and repackage the box to continue. default: default: This is not an error message; everything may continue to work properly, default: in which case you may ignore this message. ==> default: Mounting shared folders... default: /vagrant => C:/Users/Mike/VirtualBox VMs/DebianTest1 Failed to mount folders in Linux guest. This is usually because the "vboxsf" file system is not available. Please verify that the guest additions are properly installed in the guest and can work properly. The command attempted was: mount -t vboxsf -o uid=`id -u vagrant`,gid=`getent group vagrant | cut -d: -f3` vagrant /vagrant mount -t vboxsf -o uid=`id -u vagrant`,gid=`id -g vagrant` vagrant /vagrant The error output from the last command was: stdin: is not a tty mount: unknown filesystem type 'vboxsf'
- OK, so I followed this article and have pulled together these commands to try to get the Additions installed on the base box (here, I destroyed the clones and am working to repair the box from which I cloned)
sudo apt-get install gcc sudo apt-get install make
(reboot the box)
sudo mount /dev/cdrom /media/cdrom cd /media/cdrom sudo ./VBoxLinuxAdditions.run
- Sadly, this procedure did *not* repair the rsync issue that I was seeing originally before packaging from that box, but I tried vagrant package and clone from this box again, in case what I did happens to have solve the “unknown filesystem type ‘vboxsf'” error
- Nope, same problem on the cloned box – “unknown filesystem type ‘vboxsf'”
- Just in case, I tried installing the additions in the cloned box
- They won’t install – complains about the following:
Building the VirtualBox Guest Additions kernel modules The headers for the current running kernel were not found. If the following module compilation fails then this could be the reason. Building the main Guest Additions module ...fail! (Look at /var/log/vboxadd-install.log to find out what went wrong) Doing non-kernel setup of the Guest Additions ...done.
- And that log file says:
/tmp/vbox.0/Makefile.include.header:97: *** Error: unable to find the sources of your current Linux kernel. Specify KERN_DIR=<directory> and run Make again. Stop. Creating user for the Guest Additions. Creating udev rule for the Guest Additions kernel module.
- So I tried this article next, but that didn’t entirely help – the headers still aren’t available
- However, instead of erroring out, the VBox Guest Additions installation said this at the same place:
Building the VirtualBox Guest Additions kernel modules The headers for the current running kernel were not found. If the following module compilation fails then this could be the reason. Building the main Guest Additions module ...done. Building the shared folder support module ...done. Building the graphics driver module ...done. Doing non-kernel setup of the Guest Additions ...done. Starting the VirtualBox Guest AdditionsInstalling the Window System drivers Could not find the X.Org or XFree86 Window System, skipping. ...done.
- HOLY SHIT GUYS, it actually worked:
default: Mounting shared folders... default: /vagrant => C:/Users/Mike/VirtualBox VMs/DebianTest1
- So I rebuilt the base Debian box with this procedure – sadly it didn’t address the rsync issue (that was a valiant wish), but at least I should be able to clone this box and avoid the ‘vboxsf’ issue
- To address the rsync issue, I tried a variant on this article’s top answer – placed the cwrsync folder on my C:\ root, then copied the script to an rsync.bat file
Issues I never solved
- Q: what’s the source of the actual problem with the rsync on my BaseDebian box?
- TODO: Follow this article for some ideas on troubleshooting/narrowing down the problem
- Q: what’s the source of the problem that causes my “vagrant up” to fail twice on the SSH auth?
- TODO: Follow this article to see if it clears the problem
Other references
Advertisements
2 thoughts on “Windows pre-WSL: fighting with vagrant, virtualbox and Cygwin”