gnome-boxes: Introducing shared folders

In this blog post I’ll talk a little about the motivation behind shared folders in gnome-boxes and the challenges encountered while implementing it. I will also make sure to post a few UI pictures, alongside a very short guide for those of you eager to test this fresh out of the oven feature.

1. Motivation

Being able to share a directory between the host machine and a guest machine is, needless to say, a very convenient way of accessing files from one another. Thanks to the SPICE developers, an API is available which is capable of doing just that. (the only conditions are that the guest machine must have a  SPICE display and the spice-webdavd service installed). Considering this, the decision of further implementing shared folders in gnome-boxes is certainly not one to think about twice.

    2. Challenges

The first challenge was to implement *multiple* shared folders, considering that the underlying library only supports a *single* shared folder. After many discussions, we followed Marc-Andre Lureau’s suggestion of simply creating symlinks for every shared folder inside a config directory, which is the one eventually shared trough SPICE.

The second challenge was assuring the persistence of shared folders. Once the user has shared a directory, it must remain shared even after restarting the guest machine, GNOME-Boxes or the host machine. Following Felipe Borges’s suggestion, I used Gsettings and Variants, which is a great way of accomplishing tasks like this.

3. Short guide and UI pictures

Since this feature is meant to be as out-of-the-box as possible, the guide is more of a demonstration rather than a tutorial. Enjoy!

 

• 1st step: Install the spice-webdavd service in the guest machine.

spice-webdavd for Windows

spice-webdavd for Linux

The Linux version can also be installed from the official repositories.

 

• 2nd step: Play with the new Properties ->  ‘Devices & Shares’ section, in GNOME-Boxes.

Make use of the newly added UI in order to choose which folders belonging to the host machine need to be shared with the guest machine.

Note: if no shared folders are selected, the Public folder is shared by default.

• 3rd step: Access the shared folders from inside the guest machine

In case the “Spice client folder” device doesn’t show up at first, make sure to refresh the file manager.

• 4th step: Have some fun !

 

 

A big thank you to everybody who has helped me implement this feature: Zeeshan Ali, Felipe Borges, Victor Toso, Fabiano Fidencio and Cristophe Fergeau

gnome-boxes: GSoC Evaluation

This post is meant to be a final self-evaluation and self-analysis of my work for gnome-boxes during the summer. The initial project idea was about implementing/fixing a bunch of SPICE-based features/bugs to/in Boxes. The list of bugs of the SPICE component has since changed, as some new bugs have been discovered and some old ones have been closed, so I made a summary of my involvement:

Tasks completed:

Some of these. although have been completed from my point of view, still need my mentor’s final approval. I have published demo video screen captures of all the implemented features in my previous posts.

• I added a bunch of useful new help group options.

• I created a way for the user to be informed that a guest machine with SPICE display doesn’t have the SPICE agent installed. This was a problem before, because you had a hard time figuring if the SPICE agent is properly working in the guest or not.

• I made the USB redirection also available for inactive SPICE displays.

• I fixed a bug that was preventing GNOME-Boxes to close the windows of remote machines that were powered off from inside the guest.  (initial bug was about SPICE, but it also happened to remote machines with VNC display ).

• I reported and fixed a major bug, that was causing GNOME-Boxes to close entirely when powering off/pausing a live libvirt machine that had its separate window. Although the patches haven’t been accepted yet because they need more testing, my mentor told that the fix looks really good.

• I implemented  a better way of sending files to guests with SPICE display, alongside more visual feedback regarding the transfers. I have created loading bars which users will be able to see during one/multiple transfer and they will also have the ability to cancel transfers. In addition, an extra screen will also appear above the SPICE display which will inform the users that they are about to initiate a transfer.

• I made use of the already existent spice-gtk API and implemented folder sharing between the host and a guest with SPICE display. I also created patches for making it as much out-of-the-box as possible (the WebDAV channel is automatically added to local displays and spice-webdavd is automatically installed in windows/fedora). Stuff still needs to be done, which I will cover in the next section of this post.

• I managed to expose SPICE/VNC connections on the network. Through the help of some libvirt-gconfig API I added, the user will be able to expose connections of local displays on the network at choice.

• I made GNOME-Boxes able to create remote machines out of SPICE/VNC connections exposed on the network by other GNOME-Boxes clients.

• I added two new phases to the remote machines wizard, setup and preparation, which now test the connection and check that the user has the right credentials to access the remote machine.

 

 

Future goals:

• Multiple shared folders:

So far I have created the UI for multiple shared folders in Boxes and  got to know a lot about how things work by implementing multiple shared folders in phodav. Marc-Andre Lureau suggested a better way of implementing multiple shared folders, by creating links to all the folders in a GNOME-Boxes private folder, which requires work only on the client part of SPICE layers. Considering that phodav will be reimplemented soon, this is definitely the best solution.

• Connect to remote libvirt machines using SSH

So far I have done some research and came up with a solution to the problem in libvirt, by creating a SSH tunnel and providing using a method call a fd through which the connection will be possible, although I didn’t found the time to actually implement it.

Challenges:

• spice-webdavd needs to be included and mantained in Debian/Ubuntu as a package, a task which I learned that is a lot more difficult than it appears to be . I will try to involve as much as possible to help this.

I have also made a spreadsheet with links to all the patches:

spreadsheet

 

A shout out to my mentors: Zeeshan Ali, Victor Toso, Pavel Grunt and Fabiano Fidencio for the much needed help and advices 🙂

 

 

 

gnome-boxes: Coder’s log 3

July, 24th – August, 8th

For the ones that aren’t avid readers of my blog or just missed out on my last blog post, two weeks ago I talked about the fact that I managed to implement connection sharing in GNOME-Boxes.

As a follow up, some underlying mechanisms have changed, as I introduced two new classes to libvirt-gconfig which represent the hardware ability of a display to be advertised on the network (an abstract class which represents a listen child node
of the graphics device and a derived class which represents a listen child node of type address). This change of plans happened because of the need to only support the preferred XML format for a machine, namely the graphics device’s newly added listen nodes and not the old listen attribute.

I did my best to further refine the UI of the new feature, to try to get it rid of any potential bugs and to organize and improve the 20+ patches as best as possible (although I rushed a little at the end, because it felt like a never ending task and I was very excited to start further implement other stuff 🙂 ).  I won’t post another video with this feature for now, but don’t worry as I will definitely post a demonstration of it when it reaches its final form.

 

Last blog post I also talked about starting working on the SPICE layers. I won’t try to pose as the brave hero, as I was a little intimidated at first by the multitude of components working together. After quickly figuring out how the SPICE layers work, I realized that I had nothing to be afraid of and accomplishments also came along.

Firstly, let’s talk about my strategy of implementing multiple shared folders on top of the already implemented single shared folder. I am planning to make the SPICE session handle as many shared folders as the number of WebDAV channels present in the XML (analogous to the way multiple USB devices redirection is implemented).  This has one disadvantage, the fact that after adding more shared folder than previously provided, the machines needs a restart. However, the user would only need a restart when increasing the number of shared folders comparing to the previous configuration and  this can be avoided  by enabling by default a decent number of shared folders. Advantages are more numerous and include intuitiveness, simplicity and making more use of the existing, tested code.

Now, for the progress. The first step was to change the way the  SPICE server detects WebDAV channels, in order to be able to create WebDAV channels with different port names than the current default one, “org.spice-space.webdav.0”.

The second step was to modify phodav’s spice-webdavd server, to support listening for multiple  shared folder connections. In order to achieve this, I took some baby steps:

• I did a much needed code refactoring, as the code didn’t appear to be designed for multiple shared folders.
• I encapsulated the data that would represent one shared folder server/connection in a ‘Channel’ structure and further changed a lot of the method’s signatures, for them to be able to operate on a specific instance of a ‘Channel’.
• As a final baby step, I added a method that can count the available number of WebDAV channels, in order to only listen for the right number of connections.

I then tested it using two channels and, after some seg faults and a few headaches, it worked 🙂 .

Although multiple connections establish successfully, the client part of SPICE isn’t yet prepared to handle multiple shared folders, so both receive the same folder to access. After reading so much text until now, I’m sure you deserve a picture:

 

Further steps are preparing the client layer of SPICE to handle the multiple shared folders and to integrate the whole feature with Boxes. This probably won’t take a lot of time and I could also start working on using SSH to connect to the SPICE display of a remote libvirt box ( https://bugzilla.gnome.org/show_bug.cgi?id=751580 ).

gnome-boxes: Coder’s log 2

July, 10th – July. 24th

I will be making a more visual blog post this time,  filled more with imagery rather than text. Why ? Because I managed to create some patches for GNOME-Boxes that can be shown off, and why not brag a little when you have what to brag with ? 🙂 So let’s leave the worries and hardships for the end of this post and just get a taste of the soon-to-be new features of Boxes.

Disclaimer: This awesome images/videos are just a peak at the future of GNOME-Boxes and may still be subject to change into something even more awesome until the patches get merged with the project.

Here is an example of boxes creation out of a published remote connection. For demonstration purposes, the published machine is on the same host as the client, but it works exactly the same when publishing from a different host on the local network.

In the first part of the video, we expose a display connection, so that it can be reachable on the network. This is the server part of the feature.

In the second part, we check for available connections and, after selecting one of it, we create a box out of it.

Video here:

 

For the sake of not letting the user create useless boxes, I added two new phases to the wizard for remote machines.

The first one, the ‘Preparation’ phase, checks whether the remote connections is actually reachable and that we can connect to it. You will need to have faith in its existence, because testing the connection usually happens so fast that we can only see this phase when connecting to the display wasn’t successful.

The second one, the ‘Setup’ phase, checks whether the user has the correct credentials to login into the remote machine. Once the credentials are inserted, they are saved as part of the box property.

I have also prepared a video to show as much as possible both of these phases. They are implemented for SPICE and VNC remote machines.

 

On my last report, I mentioned I was stuck at creating a drag overlay for SPICE displays. After some trial and error I came up with a solution that doesn’t require any workaround, so I can say I was worrying for nothing. Here’s the result:

 

I also remade the folder sharing feature to fit the existing SPICE API. After some discussions with my mentor, Zeeshan Ali, this will at most be a temporary solution, as I will hopefully implement multiple shared folders in spice-gtk and further implement multiple shared folders in Boxes.

So there you have it. Stuff you can show to the world from your work. Awesome feeling.

Feel free to give me feedback or advice on viorel.visarion@gmail.com.

Now, for future plans, I will probably be working a lot on the SPICE layers. I will try to implement multiple shared folders in SPICE and to make wifi-geolocation work inside VMs ( bugzilla link ) .

So, that’s it . Another blog post will follow two weeks from now, hopefully  with many good news . 🙂

Avahi browsing example in Vala

After recently posting an example of publishing services in Vala using Avahi, let’s try to paint the whole picture by posting an example of  browsing those services. Enjoy !

using Avahi;
public class AvahiBrowser {
    private const string service_type = "_demo._tcp";

    private Client client;
    private MainLoop main_loop;
    private List<ServiceResolver> resolvers;
    private ServiceBrowser service_browser;

    public AvahiBrowser () {
    main_loop  = new MainLoop ();
    try {
        service_browser = new ServiceBrowser (service_type);
        service_browser.new_service.connect (on_new_service);
        service_browser.removed_service.connect (on_removed_service);
        client = new Client ();
        client.start ();
        service_browser.attach (client);
        resolvers = new List<ServiceResolver> ();
        main_loop. run ();
     } catch (Avahi.Error e) {
     warning (e.message);
    }
}

public void on_found (Interface @interface, Protocol protocol, string name, string type, string domain, string hostname, Address? address, uint16 port, StringList? txt) {
    print ("Found name %s, type %s, port %u address%s\n", name, type, port, address.to_string ());
}
public void on_new_service (Interface @interface, Protocol protocol, string name, string type, string domain, LookupResultFlags flags) {
    ServiceResolver service_resolver = new ServiceResolver (Interface.UNSPEC,
                                                            Protocol.UNSPEC,
                                                            name,
                                                            type,
                                                            domain,
                                                            Protocol.UNSPEC);
                                                            service_resolver.found.connect (on_found);
    service_resolver.failure.connect ( (error) => {
        warning (error.message);
    });

    try {
        service_resolver.attach (client);
    } catch (Avahi.Error e) {
        warning (e.message);
    }

    resolvers.append (service_resolver);
}

public void on_removed_service (Interface @interface, Protocol protocol, string name, string type, string domain, LookupResultFlags flags) {
    print ("Removed service %s, type %s domain %s\n", name, type, domain);
}

static int main (string[] args) {
    AvahiBrowser browser = new AvahiBrowser ();

    return 0;
}
}

For any questions or for any suggestions on improving this example, feel free to contact me at viorel.visarion@gmail.com.

gnome-boxes: Coder’s log

 

June, 27th-July 9th

So another two weeks have passed and it’s time to sum things up and reflect a little on the struggles and accomplishments that have marked this time period, which was quite a bumpy  ride compared to the others, but definitely more exciting.

Good news first, I have managed to make gnome-boxes successfully advertises SPICE connections on the local network and to browse already existent connections, although the UI is not yet implemented and there is still work to do.

Hopefully very soon, GNOME-Boxes will be able to create new “Boxes” out of advertised SPICE connections in a very straightforward manner. Although there is still a decent amount of work to do, I already have developed strong feelings for this awesome feature, so the future sounds very bright for it :).

Now, to talk about the shared folder feature, I managed to make GNOME-Boxes automatically add the WEBDAV-channel to every virtual machine and I am very close to making Boxes install spice-webdavd after any express installation. To translate this, the shared folders is going to be an out of the box feature, which will in most cases require no configurations from the user.

As for the bad news, well, now that I think about it, there aren’t any bad news per se, but adding a visual display when someone hovers with a file over a machine is proving to be a little more difficult task than expected, because the new UI component ends up on top of our machine’s display and replaces its drag destination site, so it will most likely require some workarounds.  Here is an example of the intended UI:

 

All in all, the light at the end of the tunnel is a ton more visible now for some features, some tiny bugs have been fixed and the future of Boxes looks very interesting and exciting !

 

 

 

 

Avahi publishing example in Vala

I should first of all say that I have seen a handful of Python examples for publishing services, some C ones, but none written in Vala, so I’m guessing this could be useful for some of you.

It is quite intriguing the fact that the API for publishing services is significantly different in Python compared to Vala, so it’s not straightforward at all translating the code from one language to the other.

Without further ado, the mighty code:

using Avahi;
public class AvahiPublisher {
    private string domain = "";
    private string host = "";
    private EntryGroup group = null;
    private Client client;
    private MainLoop main_loop;
    private List<EntryGroupService> services;

    public AvahiPublisher () {
        main_loop  = new MainLoop ();
    try {
        client = new Client ();
        client.state_changed.connect (on_server_state_changed);

        client.start ();

        group = new EntryGroup ();
        group.state_changed.connect (entry_group_state_changed);

        group.attach (client);

        } catch (Avahi.Error e) {
            warning (e.message);
        }
    }

    public void add_service (string service_name, string service_type, uint16 service_port, string? service_text) {
        try {
            var service = group.add_service_full (Interface.UNSPEC, Protocol.UNSPEC, (PublishFlags) 0,
                                                  service_name, service_type, domain, host, service_port,
                                                  service_text);
            services.append (service);
        } catch (Avahi.Error e) {
            warning (e.message);
        }
    }

    public void publish () {
        try {
            group.commit ();
            main_loop.run ();
        } catch (Avahi.Error e) {
            warning (e.message);
        }
    }

    public void unpublish () {
        try {
            group.reset ();
        } catch (Avahi.Error e) {
            warning (e.message);
        }
    }

    public void entry_group_state_changed (EntryGroupState state) {
        if (state == EntryGroupState.ESTABLISHED) {
            print ("Services established\n");
        } else if (state == EntryGroupState.FAILURE)
            print ("Services failed to establish\n");
        }

    public void on_server_state_changed (ClientState state) {
        if (state == ClientState.S_COLLISION)
            print ("Server collision\n");
        else if (state == ClientState.S_RUNNING)
            print ("Server running\n");
    }

    static int main (string[] args) {
        AvahiPublisher publisher = new AvahiPublisher ();
        string service_name = "Demo Service";
        // See http://www.dns-sd.org/ServiceTypes.html
        string service_type = "_demo._tcp";
        uint16 service_port = 1234;
        string service_text = "Something crazy and sassy.";

        publisher.add_service (service_name, service_type, service_port, service_text);
        publisher.publish ();

        return 0;
    }
}

I am currently using Avahi for properly exposing SPICE connections of local libvirt-machines in Boxes on the network, so there certainly will be a follow-up post regarding my progress.

For any questions or for any suggestions on improving this example, feel free to contact me at viorel.visarion@gmail.com.