Advanced search

Message boards : News : do you have any experience with running a VM based application in BOINC?

Author Message
Profile GDF
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Send message
Joined: 14 Mar 07
Posts: 1957
Credit: 629,356
RAC: 0
Level
Gly
Scientific publications
watwatwatwatwat
Message 42416 - Posted: 16 Dec 2015 | 23:55:12 UTC
Last modified: 16 Dec 2015 | 23:55:40 UTC

There are projects giving out VMs instead of normal applications. We are considering it. How is it? Does it work?

thanks.

gdf

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 1576
Credit: 5,606,436,851
RAC: 8,676,721
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42417 - Posted: 17 Dec 2015 | 0:22:21 UTC - in response to Message 42416.

The people to ask would be the guys at CERN. They developed the technology and the hooks into the BOINC framework. See projects like Virtual LHC@home, ATLAS@Home, and CMS-dev.

Their particular solution (I'm sure others are possible)

1) Runs a single BOINC meta-task for a fixed period, typically 24 hours. This task launches the virtual machine, and the VM fetches the tasks to be computed and returns the results via IP tunneling. This requires specialist project servers and schedulers, and if they run out of work, the computing resources can't be released to other projects and remain idle. I find this mode of working less satisfying to the interested, involved, volunteer.

2) The CERN infrastructure (the only one, AFAIK, supported by BOINC) relies on the Oracle VirtualBox technology. This is open source (i.e. free), but it doesn't - yet - support 'VGA passthrough': in other words, GPUs can't be utilised inside the virtual machine. That may be a showstopper for this project.

Profile Raptures Riot
Send message
Joined: 30 Apr 11
Posts: 6
Credit: 220,588,795
RAC: 0
Level
Leu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42418 - Posted: 17 Dec 2015 | 3:49:11 UTC - in response to Message 42416.

With the limitations Richard has noted, my personal experience is:
1) The volunteer must commit to running VMs when they create an account.
2) VMs run normal priority so they tend to crowd out other polite project tasks (and the user) if running near CPU or memory limits.
3) VMs, of course, reserve resources up front (particularly memory) and don't share it with the user.
4) VM's use resources just to build the VM. It would be nice if a constructed VM could be used over and over once allocated.

None the less, I've committed to running for CERN because I'm interested in their research.

RaymondFO*
Send message
Joined: 22 Nov 12
Posts: 72
Credit: 14,040,706,346
RAC: 0
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42419 - Posted: 17 Dec 2015 | 5:06:20 UTC - in response to Message 42418.

I briefly crunched over at Atlas@home ("Atlas") with the BOINC Virtual Box ("VB") and IMHO like all software emulation packages, this has its limitations. The software emulation is good, however when you are dealing with very complex and/or demanding tasks it usually does not work as well and is subject to higher than normal errors and a certain amount of crunching frustration. Specifically with Atlas, one had to run those tasks with hyper-threading turned off, the CPU must be powerful, it was difficult to run another projects tasks side-by-side, tasks were prone to higher than normal failure rates especially when the task says it is at 100% completed and is still running for over 24 hours for an non hyper-threaded CPU (Intel 3930k, 16gig RAM running four (4) Atlas tasks concurrently. That one lasted over 48 hours before I aborted). This is besides what I believe is Atlas project very poor point yields for completed tasks that are unacceptably low (less than 20 points per hour and I pull the computer from the project. Others usually will have different standards, if any).

VB does sometimes work well, however software emulation just does not hold up well with very demanding and complex tasks as Atlas has proven. VB is well suited for more light work type of tasks. The best analogy I have is this, running VB (BOINC version) in any OS environment is almost the same as running a high definition native PS 4 graphic game (non-ported or optimized version for that OS) using a Windows OS (7,8, or 10) emulation software. Yes it will run, but not all that well and over time you may get very frustrated with the overall performance.

I personally will not run any BOINC VB task and have pointed out the risks to other cruncher's about these types of tasks, however their are many who will still accept and crunch these tasks for the sake of the science being generated.




Profile Conan
Send message
Joined: 25 Mar 09
Posts: 25
Credit: 582,385
RAC: 0
Level
Gly
Scientific publications
wat
Message 42420 - Posted: 17 Dec 2015 | 8:06:26 UTC

I believe that Cosmology@Home has recently also added a VM version on to their project, running the original single thread CPU application along side the VM application, so they might be a useful to call.

Also RNAWorld has been using VMs for a long time now.

CPU only of course.

But although I have not run VMs before I also read (same as Richard and others) that the GPU is ignored by the VM, you can use all CPU cores (multi-thread) but not the GPU.

Conan

Profile nenym
Send message
Joined: 31 Mar 09
Posts: 137
Credit: 1,308,230,581
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42421 - Posted: 17 Dec 2015 | 8:47:54 UTC

I have tried to crunch some VM projects without any success. VM client didn't receive any work due to LAN, modem and firewall configuration. Standard Boinc applications runs without any problem.

Matt
Avatar
Send message
Joined: 11 Jan 13
Posts: 216
Credit: 846,538,252
RAC: 0
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42422 - Posted: 17 Dec 2015 | 12:10:17 UTC

I have never had any success attempting to run projects which use a VM.

Paracelsus
Send message
Joined: 11 Aug 10
Posts: 11
Credit: 21,424,870
RAC: 0
Level
Pro
Scientific publications
watwatwatwatwatwatwatwatwat
Message 42423 - Posted: 17 Dec 2015 | 12:16:31 UTC

A CERN authored 2012 paper, "BOINC service for volunteer cloud computing" may be requested through researchgate.net which broadly outlines the integration of CernVM with BOINC:

https://www.researchgate.net/publication/258668690_BOINC_service_for_volunteer_cloud_computing

Regarding the general issues of hypervisors (VB or other) used for distributed computing,the drawbacks mentioned in the thread (GPU pass through, performance, resource usage and overhead etc.) might to some degree be addressed by using containers rather than full visualization hypervisors. I might be interesting to explore delivering the BOINC client as a container with GPU pass though.

[VENETO] boboviz
Send message
Joined: 10 Sep 10
Posts: 158
Credit: 388,132
RAC: 0
Level

Scientific publications
wat
Message 42424 - Posted: 17 Dec 2015 | 13:09:25 UTC - in response to Message 42420.

I believe that Cosmology@Home has recently also added a VM version on to their project, running the original single thread CPU application along side the VM application, so they might be a useful to call.


Cosmology uses VM and
Docker

Chico Bobo
Send message
Joined: 22 Oct 14
Posts: 1
Credit: 27,417,457
RAC: 0
Level
Val
Scientific publications
watwatwatwatwatwat
Message 42427 - Posted: 17 Dec 2015 | 16:26:03 UTC

I will let others more wise in the ways of the VM environment speak to the technicalities. I will comment on being an end-user. I have a very low success rate of using VM for projects. They, more often than not, seem to hang with messages like failed to enter in a timely fashion, etc. I haven't seen much help in the forums. Every once in a while one runs to completion yet I do not know what is different in my computing environment. I do run a lot of projects so maybe it has something to do with the allocation of resources - but I am uncertain at this point. I usually abort and restart the programs. Sometimes I can get them to run (longer) if I exit BOINC and/or restart my PC but that is a little bothersome.

captainjack
Send message
Joined: 9 May 13
Posts: 171
Credit: 2,323,654,288
RAC: 2,351,655
Level
Phe
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42430 - Posted: 17 Dec 2015 | 18:59:15 UTC

Thought I would go ahead and add my 2 cents worth.

The advantage of a virtualbox based application is that the researchers only have to make their application work with one guest operating system. CERN uses a version of Linux that they optimized for their type of scientific computing.

And here are the disadvantages that I see:

Running a task under an operating system on a virtual machine suffers about a 15% performance degradation.

For each task running, there has to be a copy of the virtualbox guest operating system and the scientific task loaded into memory. CERN tasks take between .5 GB and 2.5 GB memory for each task. Most typical PC's can only run one or two tasks at a time because of the memory requirements.

The BOINC task in the host operating system invokes a vboxwrapper that in turn initiates the guest operating system in virtualbox. Finding a working combination of the technology stack (host operating system, vboxwrapper, version of virtualbox, guest operating system and research task seems to be challenging. There seems to be quite a bit of time spent getting everything to work together again when any of the components of the technology stack upgrade their software to a new major release.

The VM application seems to suffer when the host hardware stops abruptly, for example a electrical outage. Often the BOINC task has trouble communicating with virtualbox to get the VM started up again. This quite often leads to aborted tasks and starting up another task.

Because of the challenges in getting everything to work together properly, the CERN projects that use VM get a low participation rate. The Test4theory (vLHC@home) project usually has between 500 and 700 participants per day. And because of the memory restrictions, most of those users are only running a few tasks at a time. I have 36 threads available in my crunching farm and usually run 3-4 VM tasks at a time.

Having said all that, I do run tasks for CERN. But I sure don't get it to run by being a first adopter whenever one of the components in the technology stack comes out with a new version. I wait until the researchers have some time to experiment and get everything to work together properly.

Hope that helps.

xixou
Send message
Joined: 8 Jun 14
Posts: 18
Credit: 19,804,091
RAC: 0
Level
Pro
Scientific publications
watwatwat
Message 42431 - Posted: 17 Dec 2015 | 20:54:12 UTC

I do not like the tasks submitted to the virtual box,
sometime the task cannot start (syncro issue with the linux tasks).

Jim1348
Send message
Joined: 28 Jul 12
Posts: 819
Credit: 1,591,285,971
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42432 - Posted: 17 Dec 2015 | 22:37:00 UTC

I have two Haswell machines running Win7 64-bit and VirtualBox 5.0.10 (BOINC 7.6.21), and generally like it a lot. As captainjack mentioned, it allows the researchers to optimize their application on whatever version of Linux (or anything else I suppose) that they want, and it all works more or less the same on each client. Much of the hassles on each project involve trying to optimize the software for some particular OS/Hardware combination from what I can see.

I have one machine on CERN (Atlas, vLHC, CMS-dev) and have no problems. With the new wrapper for Atlas, my error rate is now zero, as it has been for the other projects from the beginning. However, you do need a LOT of memory in some cases, so don't go into it unprepared.

The other machine is on Cosmology, and as noted above, that is multi-threaded. That can cause some problems in sharing cores among projects, since Cosmology wants to grab them all. I can devote an entire machine to it (reserving a couple of CPU cores for GPU work), but most people will get rather annoyed at it and leave the project, so I don't really recommend it for most projects.

As for GPUs, I was under the impression that the new Version 5 of VBox allowed for that, but maybe it is not implemented yet. I think it is supposed to allow for SSE and AVX also, but have not seen that used yet. I would try it if possible.

Profile [AF>Amis des Lapins] Phil...
Send message
Joined: 16 Jul 13
Posts: 56
Credit: 1,626,354,890
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42433 - Posted: 17 Dec 2015 | 22:45:57 UTC

http://www.cosmologyathome.org/forum_thread.php?id=7348&postid=20629

mikn
Send message
Joined: 17 May 14
Posts: 1
Credit: 54,433,253
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwat
Message 42434 - Posted: 17 Dec 2015 | 23:14:42 UTC

Works for me, Atlas and LHC run well, even on antique laptops, regards mikn

BelgianEnthousiast
Send message
Joined: 7 Apr 15
Posts: 33
Credit: 1,201,157,375
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwat
Message 42435 - Posted: 17 Dec 2015 | 23:15:44 UTC

Hi GDF,

I'm running both Atlas and LHC@Home (CERN) as some people below do as well.
My experience with VM's is that they are very stable as also pointed out by Jim1348, once they are stabilized.

I've run them now for nearly 8 months nearly uninterrupted and did not encounter major issues so far. (knocking on ...)

A couple of drawbacks that I see are:
1) Consuming a lot of memory (Atlas is now running 5 simultaneous tasks and EACH task takes up between 5.7 GB and 7.2 GB) I got 32 GB of RAM so I'm still fine, but on other systems with lower memory, that could become fast prohibitive.

2) Atlas and LHC tend to be VERY disk intensive with up to 80 GB per day being written. I have raised the point multiple times, but without satisfactory answer.

3) As you're working with VM's, you need to reserve a core only for the VM ware (whether it's Oracle VirtualBox or VMWare, or whatever)
If you want to use your PC still, you'll need to take off another core for your applications as well as the BOINC Manager itself.

4) For each core, you'll also take a 15 % hit (depends on processor type & speed) on performance due to the emulation going on.

So, it definitely has it's advantages, but it also has some important drawbacks, so you'll need to carefully weigh your options and think of your target audience.

Hope this helps ?

Kind Regards,
K.

Jim1348
Send message
Joined: 28 Jul 12
Posts: 819
Credit: 1,591,285,971
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42448 - Posted: 17 Dec 2015 | 23:32:58 UTC - in response to Message 42447.

2) Atlas and LHC tend to be VERY disk intensive with up to 80 GB per day being written. I have raised the point multiple times, but without satisfactory answer.


Good point. I use a large (12 GB) write cache (PrimoCache) to protect my SSD, and it apparently helps to reduce the error rate also, and reads can come out of that cache also.

With a GPU project like GPUGrid, writes may not be a problem though.

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 1576
Credit: 5,606,436,851
RAC: 8,676,721
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42450 - Posted: 18 Dec 2015 | 0:23:33 UTC - in response to Message 42432.

As for GPUs, I was under the impression that the new Version 5 of VBox allowed for that, but maybe it is not implemented yet. I think it is supposed to allow for SSE and AVX also, but have not seen that used yet. I would try it if possible.

Searching https://www.virtualbox.org/wiki/Changelog for either VGA or passthrough doesn't reveal much. AVX and AVX2 have been disabled (at v5.0.2) and not yet restored. Webcams are supported. And that's about it.

Jazzop
Send message
Joined: 15 Aug 10
Posts: 1
Credit: 1,624,760
RAC: 0
Level
Ala
Scientific publications
watwat
Message 42451 - Posted: 18 Dec 2015 | 7:39:47 UTC

Another problem with VM projects: They don't play nicely with Hyper-V. I cannot/will not run any Virtualbox project on my machines that are Hyper-V hosts. This means that I have one or two isolated computers (both laptops) that will crunch the occasional Vbox project. Furthermore, I have experienced a ridiculously high rate of problems (most of which are mentioned already in this thread) from those projects, so I often set them to "no new work" until I am bored enough to troubleshoot them.

Please don't go there.

Profile Acey Pilot
Send message
Joined: 4 Jan 14
Posts: 4
Credit: 1,901,240,470
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwat
Message 42452 - Posted: 18 Dec 2015 | 13:56:03 UTC - in response to Message 42416.

There are projects giving out VMs instead of normal applications. We are considering it. How is it? Does it work?

thanks.

gdf



I've tried, but find it uses too much memory and is very CPU intensive, so quit. Some searches will verify this for you. Personally, I think GPUs is the way to go, because of efficient energy use.

Profile Tiger
Send message
Joined: 30 Jan 15
Posts: 7
Credit: 402,017,837
RAC: 0
Level
Gln
Scientific publications
watwatwatwatwatwatwatwat
Message 42454 - Posted: 18 Dec 2015 | 20:22:25 UTC

After dealing with a BIOS configuration glitch at the beginning, I have VB running VirtualLHC and Atlas since June 1st, 2015. I went to the forum and got the info I needed to get her up and running. A couple glitches after Windows 7 upgrades, but then back to running smooth after a couple days without touching it.
____________

Gennady Fedonin
Send message
Joined: 17 Nov 07
Posts: 1
Credit: 22,981,076
RAC: 0
Level
Pro
Scientific publications
watwatwatwat
Message 42455 - Posted: 18 Dec 2015 | 23:16:06 UTC - in response to Message 42416.

Tried some. Not working properly on my machine...

Silverdrake
Send message
Joined: 28 Nov 14
Posts: 1
Credit: 8,243,728
RAC: 0
Level
Ser
Scientific publications
wat
Message 42456 - Posted: 19 Dec 2015 | 4:13:34 UTC

I have been running VM applications for RNAWorld. As others have mentioned, they have a high error rate. If the computer crashes, it can cause multiple snapshots in the VM that have to be cleaned up manually before the machine will run, again. This can also cause the WUs to error out.

The VM doesn't run well on low-end computers and is RAM intensive. My old box (now my husband's) has dual-core CPU and 3G of RAM, and the VM has to wait until the computer is completely idle to have enough RAM to run. My current box is 6-core with 8G RAM. The VM runs on one CPU with the other 5 available to other projects.

I will note that I have never successfully completed one of RNAWorld's WUs. Those that didn't die to computer failure or operator error (see duplicate snapshots above), errored out on their own. I currently have one WU running on each machine and my fingers crossed that at least one of them will actually complete. It's damned painful to have several hundreds or thousands of hours of computing time end in a failed computation.

For users with higher-end machines and enough knowledge to play with the VMs, they seem to work just fine. For the average user who just wants to contribute spare cycles to science, they're a pain in the tailfeathers.

MingyeWang
Send message
Joined: 26 Nov 12
Posts: 1
Credit: 81,962,433
RAC: 0
Level
Thr
Scientific publications
watwatwat
Message 42457 - Posted: 19 Dec 2015 | 4:43:58 UTC

VM are quite bad at using GPU power. Since GPUgrid has not been renamed to something like IntelPhiGrid yet, it doesn't sound nice to use VMs.

Personally I'm not quite comfortable with VM projects since they makes the programs less transparent to the volunteers. The app just appears like a huge black box that eats up a fixed (probably unnecessary) amount of your memory and processing power.

If you are looking for some `compile once, run (or perhaps debug?) everywhere' solution, even wine will do better at using CUDA power. The `staging area' fork of wine, wine-staging, supports using the native CUDA support found in nVidia's proprietary driver. Please note that as long as it's still in the staging area, it cannot be considered as stable. And I can't find any support for CUDA pass-thru on OS X.

Profile Steve Hawker*
Send message
Joined: 12 Feb 13
Posts: 5
Credit: 1,926,775
RAC: 0
Level
Ala
Scientific publications
watwatwat
Message 42459 - Posted: 19 Dec 2015 | 10:01:49 UTC

ATLAS has proven to be really difficult to run. The admins have gone through countless versions over many months and after all that I now find the tasks no longer run on OSX after some success.

Cosmology, by way of contrast, worked firs time and has been going great.

So VMs CAN work but as always the source material dictates.

Having said that, I'd much rather you spend what resources you have on supporting AMD cards on Linux than on VMs.

Anubis
Send message
Joined: 14 Oct 12
Posts: 1
Credit: 2,348,818
RAC: 0
Level
Ala
Scientific publications
watwatwatwatwatwat
Message 42460 - Posted: 19 Dec 2015 | 11:56:10 UTC - in response to Message 42416.

When you say VM's I take it you mean vendors of VM applications

Microsoft HyperV, VMware ESXi, Virtual box, Etc

And also depending on CPU/RAM/STORAGE how many Vm's can be created.

All these give a virtual switch so the Application can talk to the host, that then can talk on the internet.

If what's called NAT or Network Address Translation is not setup properly you wont get any access or updates on the VM, it will just sit there not doing anything, and boinc wont update.

So get the VM's Started and on host run CMD in Admin IPCONFIG/ALL to see the gateway. Note down gateway, Goto VM and open CMD, then type PING ( Gateway ). you are either going to get a response or not, if not look at your virtual switch config its not set correctly, usually with any application on a Microsoft platform anyway now sets up a rule to let the application access the internet through the firewall, play with setting until you get a response then boinc should update

Hope this helps.

Also Raptures Riot gives a good tip as well

wiyosaya
Send message
Joined: 22 Nov 09
Posts: 114
Credit: 589,114,683
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42463 - Posted: 20 Dec 2015 | 15:23:48 UTC

FWIW - I don't run any BOINC projects in VMs, but I do run one at work from time to time to support software that only runs on XP. We use Microsoft's VM, and it is noticeably slow. Some times, it fails to connect to the resources of the base OS - Win 7 - and I must restart it at least once for it to run properly. My machine at work is a Sandy Bridge machine.

I have also setup Virtual box on one of my machines at home. It was a Sandy Bridge E running a quad core, eight thread i7-3820 with 32G of ram. I wanted to set up an isolated environment for developing an Android app. I went through the procedure to clone my Windows 7 machine and turn that into a VM to run inside of Virtual Box.

I abandoned that approach when it became apparent that I would need an additional license for Win 7 in the VM. So, this is a potential problem for us end users if the OS for the project is Windows. Also, the VM, even on the Sandy Bridge E machine, was noticeably slower than the native OS and one of the things that SBE machines do well is handle memory intensive apps like VMs.

From the sound of it and speaking as a software developer, the CERN project takes the burden off of their developers and places it on the volunteer. A project that requires high maintenance will probably only appeal to the most technical of volunteers, and some may not want to devote the time to running the project.

On that note, GDF, my question to you as a software developer would be what are your reasons for considering a VM? In my opinion, the reasons for doing something are the most important. I work in an R & D department, and there is one developer there that does things because he can, often, without consulting others, and without considering the consequences. Right now, he has a significant amount of work to do because he dug himself into a hole by writing custom code for a GL toolkit that is now outdated and now uses DX. My point here is that as I see things, it is best to try to find the best overall solution when engineering software.

While it sounds like the VM solutions work for some, it also sounds like the VM solution does not work well for others. Personally, I would be adverse to running a BOINC project that depends on a VM especially if it requires more time than I have to devote to it.
____________

Wrend
Send message
Joined: 9 Nov 12
Posts: 51
Credit: 522,101,722
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwat
Message 42464 - Posted: 20 Dec 2015 | 15:27:36 UTC

I host game servers on Debian running in Virtual Box in Windows 7. I don't run BOINC in a virtual machine though, as I don't want the additional overhead, and lack of dynamic available resources and fine control.

So, if the question is if you guys are looking to run things in user space VMs, I would advise against it unless there is a specific need to. I for one wouldn't be contributing to those work units.

Cheers.
____________
My BOINC Cruncher, Minecraft Multiserver, Mobile Device Mainframe, and Home Entertainment System/Workstation: http://www.overclock.net/lists/display/view/id/4678036#

Jim1348
Send message
Joined: 28 Jul 12
Posts: 819
Credit: 1,591,285,971
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42465 - Posted: 20 Dec 2015 | 16:09:42 UTC
Last modified: 20 Dec 2015 | 16:28:15 UTC

The question of overhead should be considered, but probably favors the VM. Most projects are developed under Linux, and I usually see anywhere from a 20% to 80% improvement running Linux over Windows (I run only Win7 64-bit myself, but look at the performance numbers of completed Linux work units, etc.). I would guess on average Linux gains a 40% advantage over Windows. If the VM imposes a 15% penalty as noted above (I don't know myself), then the project still gains a lot overall, considering that the large majority of users are on Windows.

I don't see the problems that some people face; my machines run 24/7, and once the VM is set up, then they just run. And I have had no unusual problems setting up VirtualBox on CERN or Cosmology (or RNA World, but they never have work). The amount of memory used is determined mainly by the project. Even though the CERN projects take a lot of memory due to their science, Cosmology takes very little memory. I think the real question for GPUGrid is whether or when a VM will be available that allows the use of GPUs, if that is what they intend to use it for.

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 1576
Credit: 5,606,436,851
RAC: 8,676,721
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42466 - Posted: 20 Dec 2015 | 17:25:09 UTC - in response to Message 42463.

From the sound of it and speaking as a software developer, the CERN project takes the burden off of their developers and places it on the volunteer. A project that requires high maintenance will probably only appeal to the most technical of volunteers, and some may not want to devote the time to running the project.

I think that the usual balance pushing towards the VM side of the scale is that academic/scientific programmers are more comfortable programming for Linux (and perhaps their institution's particular chosen flavour of Linux), whereas the largest available group of volunteers has Windows installed.

Many projects seem to be outside their comfort zones simply developing for and maintaining the basic trinity (Linux/OS X/Windows), and if the raw science has already been developed in-house for a specific Linux implementation, then it can appear to be a valuable time-saver just to load the science into a preconfigured VM and let it run from there. In reality, it simply shifts the problem somewhere else - into the scheduling and communications regime.

Profile Scalextrix[Gridcoin]
Send message
Joined: 27 Jan 09
Posts: 34
Credit: 185,313,973
RAC: 0
Level
Ile
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwat
Message 42470 - Posted: 21 Dec 2015 | 8:52:58 UTC - in response to Message 42416.

I have not had many issues running ATLAS and vLHC, been doing so for a long time. However when they dont want to play nicely its quite hard to figure out what went wrong and you end up manually cleaning up the snapshots.

ATLAS does use a lot of RAM and you really need to setup an app_config.xml to reduce the number of concurrent tasks to prevent it runnig out of space. vLHC limit the project to two concurrent tasks and the RAM footprint is smaller.

If you were intersted in putting GPU into the VM, then many users having 1 GPU per CPU would not need to limit the number of tasks as the number of GPUs will effectively do that. Users with 4 x GPU per CPU would still probably only need approx 16GB RAM to run this in a VM.

Profile robertmiles
Send message
Joined: 16 Apr 09
Posts: 503
Credit: 727,920,933
RAC: 155,858
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42477 - Posted: 22 Dec 2015 | 22:44:05 UTC

RNA World has been running into a problem with VM workunits: Once any computer has been detected as not supporting VM, it's very difficult to get that computer to check for VM support again with the current version of BOINC. They appear to be waiting for a new version of BOINC that fixes this before they start offering more workunits.

My computers have occasionally tried to run RNA World VM workunits. None ran properly yet, due to the VM support detection problem.

Another problem they have seen: The amount of memory used is built into the application, and cannot be adjusted as needed. This often makes it reserve much more memory than it actually uses.

Looks like you should hold off on offering VM workunits until a new version of BOINC has a fix to at least the VM support detection problem. Possibly also until a new version of VM offers GPU use support.

Profile microchip
Avatar
Send message
Joined: 4 Sep 11
Posts: 110
Credit: 326,102,587
RAC: 0
Level
Asp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42478 - Posted: 23 Dec 2015 | 3:36:07 UTC

If GPUGRID starts messing with VMs, I'm out... :/
____________

Team Belgium

angler
Send message
Joined: 11 Feb 09
Posts: 2
Credit: 53,125,631
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42479 - Posted: 23 Dec 2015 | 3:51:27 UTC

VirtualLHC (LHCat2) works okay, but resource intensive - obviously vbox and having stale vm versions can result in some bloat

Jim1348
Send message
Joined: 28 Jul 12
Posts: 819
Credit: 1,591,285,971
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42480 - Posted: 23 Dec 2015 | 7:38:38 UTC - in response to Message 42478.

If GPUGRID starts messing with VMs, I'm out... :/

But if they don't generate more work, we are all out. Using VMs may allow new researchers (i.e., grad students) coming into the group to develop their projects more rapidly, using whatever standard version of Linux that they use. If they get tripped up in the difficulties of making different versions to run on various machines, it could slow down the entire process.

HA-SOFT, s.r.o.
Send message
Joined: 3 Oct 11
Posts: 100
Credit: 5,879,292,399
RAC: 0
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42481 - Posted: 23 Dec 2015 | 10:28:12 UTC

For me. I cann't use VM. I cann't install VBox over all of my machines.

Snow Crash
Send message
Joined: 4 Apr 09
Posts: 450
Credit: 539,316,349
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42482 - Posted: 23 Dec 2015 | 14:35:43 UTC

Excellent conversation!!!

I've worked through the challenges for the CERN projects and have spent more than enough time know that I'm not a fan of VM crunching in it's current form. I still crunch CERN because I like their science as much as I do GPUGrid but it really is more work than a hobby should be.

From a cruncher's perspective VM crunching is much more complex than regular BOINC crunching but I also understand as a developer it is easier to make one version of an app. Many fine points have been made regarding the efforts to crunch and some on the implementation of VM, I don't think this is an either / or situation and as an educational institution of forward thinkers you may need to explore this avenue as you have others in the past. Perhaps implementing VM for the CPU app would provide a way to explore VMs before launching wholesale for ACEMD.

I present the following issues to hopefully make any future VM implementation easier for crunchers.

Typical reasons for apps going bad, in order of occurrence
1.) Hardware crash due to power outage, too high OC, too high temps
2.) A bug in the software itself
3.) Need a particular version of a GFX driver
4.) Rarely a Linux issue with a missing lib - usually when someone moves from 32 to 64 bits.

Those issues do not go away and then there is the addition of trying to get the VM infrastructure coordinated.
1.) What flavor of VM?
2.) Is it the same flavor that comes with BOINC?
3.) Is it the same version that comes with BOINC?
4.) Do I have to download and install software XYZ?
5.) Do I need the extension pack or not (versions must match exactly)?
6.) Do I need to change BIOS settings?

Here are some additional troubleshooting areas/ cleanup tasks that are sometimes necessary when we experience crashes, hangs, and postponements.
1.) When / how do I cleanup a snapshot - I have multiple which one is it?
2.) When / how do I cleanup a VM instance - I have multiple which one is it?
3.) When / how do I cleanup a BOINC slot - I have multiple which one is it?
4.) What do I do when the task says postponed for 24 hours?

Please also consider that a cruncher will ask any / all of the above in the forums (please create a separate forums for VM) and if they do not get quick, easy, and accurate answers they will likely move on to another project. GPUGrid enjoys a fairly high level of technical literacy amongst it's volunteers but having to handle new challenges with VM may make the boards look like the project has more issues than really exist with the science applications themselves.

____________
Thanks - Steve

Jim1348
Send message
Joined: 28 Jul 12
Posts: 819
Credit: 1,591,285,971
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42483 - Posted: 23 Dec 2015 | 15:25:13 UTC - in response to Message 42482.
Last modified: 23 Dec 2015 | 15:30:30 UTC

I do see that people like to look into the VM to see what is going on, but I just let it run. And I have not had to clean up anything since upgrading to VirtualBox 5. Also, there were problems with BOINC not cleaning up slots after failed work units, but that has been eliminated with the recent versions of BOINC insofar as I know, though I don't see many failures myself for some reason (probably because I have a lot of memory and don't reboot often).
http://atlasathome.cern.ch/results.php?hostid=17032
http://lhcathome2.cern.ch/vLHCathome/results.php?hostid=85673
http://boincai05.cern.ch/CMS-dev/results.php?hostid=688

The bundled version of BOINC now comes with VirtualBox 5.0.10, a very recent version (the latest is 5.0.12, out for only 5 days). These newer versions are mainly bug fixes, and don't affect the major functionality. So you will need to upgrade your software if you go to a VM to the latest versions, but beyond that GPUGrid will have to do a trial to see how it flies. Some users may have problems, others not. They won't know the numbers until they try it out.

Nick Name
Send message
Joined: 3 Sep 13
Posts: 53
Credit: 1,533,531,731
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwat
Message 42484 - Posted: 23 Dec 2015 | 17:25:32 UTC - in response to Message 42416.

There are projects giving out VMs instead of normal applications. We are considering it. How is it? Does it work?

thanks.

gdf

I've run them at other projects and had relatively few problems. I guess I've been lucky. They certainly have a much higher failure rate than standard BOINC apps. The average user isn't going to bother with it if there are problems, they'll move on quickly.

Are we talking about replacing the current apps with VM computing, or adding VM apps?


____________
Team USA forum | Team USA page
Join us and #crunchforcures. We are now also folding:join team ID 236370!

Iamone
Send message
Joined: 28 Aug 13
Posts: 1
Credit: 205,325,555
RAC: 0
Level
Leu
Scientific publications
watwatwatwatwatwatwatwatwat
Message 42491 - Posted: 24 Dec 2015 | 20:36:10 UTC - in response to Message 42416.

I have been running vm recently rearly am just at start.
Am away for 10 days will let you know more early next year.
Have a good Xmas and new Year.
TAG

paradoxon
Send message
Joined: 20 Feb 14
Posts: 1
Credit: 7,891,310
RAC: 0
Level
Ser
Scientific publications
watwatwatwatwatwatwat
Message 42552 - Posted: 3 Jan 2016 | 17:07:59 UTC

Two major concerns A: getting GPU support in VM is very tricky busyness, if it is to be free, easy to use, easy to maintain and cross platform. I'd say it is impossible at the time. B: Especially on Linux (maybe other unix platforms too) its sometimes tricky go get boinc and VirtualBox to play along nicely. Both my gentoo box and my arch linux make trouble. Ubuntu worked fine, but Ubuntu is not the whole community. Maybe handing out docker containers? But that would put Win-Users in trouble, wouldn't it?

Jacob Klein
Send message
Joined: 11 Oct 08
Posts: 1127
Credit: 1,901,927,545
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42555 - Posted: 3 Jan 2016 | 20:18:48 UTC

GPUGrid admins:

Was GPUGrid intending to use VMs for: GPU work, or CPU work, or both? It might be helpful to know what your intentions are...

jamie
Send message
Joined: 9 Jan 16
Posts: 1
Credit: 884,675
RAC: 0
Level
Gly
Scientific publications
wat
Message 42581 - Posted: 9 Jan 2016 | 4:04:01 UTC

For CPU based work, VMs would be just fine. 99.9% of the time however, a VM cannot access certain hardware, like the GPU(s). If you want to do GPU computing, it has to be on the host system OS.

John
Send message
Joined: 10 Jan 16
Posts: 3
Credit: 214,098,870
RAC: 50,251
Level
Leu
Scientific publications
watwatwatwatwatwat
Message 42586 - Posted: 10 Jan 2016 | 14:58:07 UTC

I use two VM projects with BOINC - Atlas and VirtualLHC. Up until recently, they were problematic especially with Windows 10. They had abnormally high disk usage and kept getting postponed saying they didn't start in a timely fashion or some such. But, now that they've finally approved Virtual Box 5 (I'm currently running 5.0.12) for BOINC 7.6.22, both projects are running very smoothly and I can't tell any difference between them and non VM projects.

sven.leuschner@gmx.net
Send message
Joined: 10 Jan 16
Posts: 1
Credit: 40,500
RAC: 0
Level

Scientific publications
wat
Message 42590 - Posted: 11 Jan 2016 | 0:40:54 UTC - in response to Message 42416.

yes, like the answers before I sugget you turn towards the CERN guys. I participate in the vLHC project - and after the latest upgrade (which included finally a 5.x Virtualbox) it runs well on my machine. On the other hand, people who like to contribute will need to have more RAM. Which isnt too big an issue looking at modern pcs - my laptop has a good 16GB .... just saying.


____________

Jim1348
Send message
Joined: 28 Jul 12
Posts: 819
Credit: 1,591,285,971
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42820 - Posted: 15 Feb 2016 | 2:00:06 UTC

This is an example of how Docker is used to create a BOINC project.
http://cosmicmar.com/2016/02/14/boinc-server-docker-1/

Profile robertmiles
Send message
Joined: 16 Apr 09
Posts: 503
Credit: 727,920,933
RAC: 155,858
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 46942 - Posted: 16 Apr 2017 | 2:14:01 UTC
Last modified: 16 Apr 2017 | 2:15:31 UTC

Some, as a user only.

I found that you first need to enable VM, usually early in the boot process where you are still talking to to the BIOS or UEFI, and have not yet told it which operating system to boot (even if you only have one operating system installed). Details of how are often specific to the model of computer you have.

If you have ever tried to download a VM workunit before, you must then edit a BOINC file which records that no usable VM was found for the previous VM workunit. Retrying this check is NOT automatic for BOINC 7.6.33 and earlier; this may change for future versions. This change needs to be done while BOINC is not running.

At least one of the BOINC projects I get VM workunits from is not compatible with the 5.0.* versions of VirtualBox, so you must now upgrade to one of the recent 5.1.* versions if you're interested in that project. Restart or reboot after this upgrade.

Now start BOINC, and tell at least one BOINC project that it is allowed to send you VM workunits.

opr
Send message
Joined: 24 May 11
Posts: 7
Credit: 93,272,937
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwat
Message 46967 - Posted: 16 Apr 2017 | 14:37:27 UTC

I ran test for theory about year ago with my old pc , virtual box installed . Somehow they tend to need more attention , especially with older pc. Little more tricky than usual "participate and forget"-style crunching.

Lluis
Send message
Joined: 22 Feb 14
Posts: 26
Credit: 672,639,304
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwat
Message 46970 - Posted: 16 Apr 2017 | 19:30:09 UTC

Two years ago I did some crunching on VM without problems until I decided upgrade to a newer version of VM. Probably I did a mistake in the process and since then it has been impossible to install VM as always a missing archive is reported. I found others users of VM with the same problem and tips of how to fix it, but in my case it didn't work.

Jim1348
Send message
Joined: 28 Jul 12
Posts: 819
Credit: 1,591,285,971
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 46971 - Posted: 16 Apr 2017 | 22:48:12 UTC - in response to Message 46967.

I ran test for theory about year ago with my old pc , virtual box installed . Somehow they tend to need more attention , especially with older pc. Little more tricky than usual "participate and forget"-style crunching.

That is quite true for the CERN projects in general, but I think that is mainly because they are at the cutting edge of science and are constantly changing both the data and the applications they run. Also, they seem to have a lot of servers that need to be working together, and they don't always do so.

It could be different on GPUGrid, if they stick with a stable application and don't change it without adequate testing. But the LHC people say that it is hard to troubleshoot the VM projects, apparently because the virtual machine itself gets in the way. But it could work OK under the right conditions.

Erich56
Send message
Joined: 1 Jan 15
Posts: 1090
Credit: 6,603,906,926
RAC: 18,783,925
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwat
Message 46974 - Posted: 17 Apr 2017 | 5:01:19 UTC

what should be considered though is that VMs require virtualization capability of the CPU; older CPUs don't have this.

Jim1348
Send message
Joined: 28 Jul 12
Posts: 819
Credit: 1,591,285,971
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 46977 - Posted: 17 Apr 2017 | 14:35:40 UTC - in response to Message 46974.

It is not a problem that I can see, except that people should be made aware that it will not work on older CPUs so that they don't waste time trying. But their GPU work units don't work on all cards either. You just don't run it in that case.

Profile robertmiles
Send message
Joined: 16 Apr 09
Posts: 503
Credit: 727,920,933
RAC: 155,858
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 46992 - Posted: 18 Apr 2017 | 2:14:15 UTC - in response to Message 46970.
Last modified: 18 Apr 2017 | 2:15:24 UTC

Two years ago I did some crunching on VM without problems until I decided upgrade to a newer version of VM. Probably I did a mistake in the process and since then it has been impossible to install VM as always a missing archive is reported. I found others users of VM with the same problem and tips of how to fix it, but in my case it didn't work.

Did you try uninstalling VirtualBox, restarting, then installing it again? That often helps with missing file problems.

Lluis
Send message
Joined: 22 Feb 14
Posts: 26
Credit: 672,639,304
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwat
Message 47020 - Posted: 18 Apr 2017 | 18:21:27 UTC - in response to Message 46992.

Did you try uninstalling VirtualBox, restarting, then installing it again? That often helps with missing file problems.


The problem is that when I try to uninstall VirtualBox appears this message or a similar one: "VirtualBox-4.3.12-r93733-MultiArch_amd64.msi" is NOT FOUND (In a TEMP directory) and I can't continue the process of uninstalling.
A tip in internet explains how to overrule this problem and some guys said that it works. But not for me.

Jacob Klein
Send message
Joined: 11 Oct 08
Posts: 1127
Credit: 1,901,927,545
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 47021 - Posted: 18 Apr 2017 | 18:45:17 UTC - in response to Message 47020.
Last modified: 18 Apr 2017 | 18:45:52 UTC

Did you try uninstalling VirtualBox, restarting, then installing it again? That often helps with missing file problems.


The problem is that when I try to uninstall VirtualBox appears this message or a similar one: "VirtualBox-4.3.12-r93733-MultiArch_amd64.msi" is NOT FOUND (In a TEMP directory) and I can't continue the process of uninstalling.
A tip in internet explains how to overrule this problem and some guys said that it works. But not for me.


Have you tried re-running the original installer for that exact version, and then choosing to uninstall during the installer, or maybe repair, or maybe re-install ? Try all the options, using the original installer.

Lluis
Send message
Joined: 22 Feb 14
Posts: 26
Credit: 672,639,304
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwat
Message 47024 - Posted: 19 Apr 2017 | 0:24:07 UTC - in response to Message 47021.

Thanks for your help.

Have you tried re-running the original installer for that exact version, and then choosing to uninstall during the installer, or maybe repair, or maybe re-install ? Try all the options, using the original installer.


That's the suggestion I've been talking about. Usually it makes sense and sometimes works fine, but in my case, when I try to install always I get the same message whether I use the original version or a new one.

Jacob Klein
Send message
Joined: 11 Oct 08
Posts: 1127
Credit: 1,901,927,545
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 47025 - Posted: 19 Apr 2017 | 1:00:24 UTC - in response to Message 47024.
Last modified: 19 Apr 2017 | 1:11:59 UTC

When using the original installer ... Do you get any options for "Repair" or "Uninstall" or "Install"? Please try to be a bit more descriptive, with exact details, on what you've tried. :)

ALSO

If you can't get the original installer "options in the UI" to work, try this:
- Run the original installer (but don't click anything in it) (I found it here: https://www.virtualbox.org/wiki/Download_Old_Builds_4_3_pre24 )
- Open File Explorer, browse to: C:\Users\{User_Name}\AppData\Local\Temp
- See if you have a "VirtualBox" folder there. If so, copy it to your desktop. It should have the .msi file you needed
- Exit the installer
- Reattempt to uninstall, and if you get prompted for the .msi file, try to have it browse to find it in the folder on your desktop

Any help?

Lluis
Send message
Joined: 22 Feb 14
Posts: 26
Credit: 672,639,304
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwat
Message 47035 - Posted: 19 Apr 2017 | 18:19:14 UTC - in response to Message 47025.

... try this:
- Run the original installer (but don't click anything in it)
- Open File Explorer, browse to: C:\Users\{User_Name}\AppData\Local\Temp
- See if you have a "VirtualBox" folder there. If so, copy it to your desktop. It should have the .msi file you needed
- Exit the installer
- Reattempt to uninstall, and if you get prompted for the .msi file, try to have it browse to find it in the folder on your desktop


It has worked. Now I am going to wait for the finish of the ongoing tasks to install VM .
I thought I had tried this trick before but surely I did not do well. Following your instructions has been very easy.
Many thanks.

Jacob Klein
Send message
Joined: 11 Oct 08
Posts: 1127
Credit: 1,901,927,545
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 47036 - Posted: 19 Apr 2017 | 19:39:28 UTC

You're welcome. I have quite a bit of VirtualBox experience in Windows.

Profile robertmiles
Send message
Joined: 16 Apr 09
Posts: 503
Credit: 727,920,933
RAC: 155,858
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 47651 - Posted: 23 Jul 2017 | 0:55:58 UTC
Last modified: 23 Jul 2017 | 1:01:47 UTC

I'm currently running a rather long RNA World VM task on one of my computers. 38 days elapsed so far, currently expected to run for another 60 days according to BOINC Manager. Its behavior suggests that the total run time will be closer to 120 days, though.

The 5.0.* versions that can be downloaded with BOINC do not work well; expect to have to install a 5.1.* version to make it work well. I'm currently using 5.1.22 on the computer where it is in use and 5.1.18 on my other computer.

These task cannot run simultaneously with an MT (multi-threaded) task from Cosmology@Home. BOINC Manager must suspend one to run the other, so usually both run mostly in priority mode.

One of the input files must be a light-weight operating system, usually Linux, set up to run in the VM environment. However, this file rarely needs to change for other RNA World tasks, so downloading it once per client computer should be enough.

Checkpoints are rather large, close to 700 MB each. Normally, only one is kept, so a failure while writing a checkpoint often does not leave any usable checkpoint.

Applications that run inside the VM appear to be 32 bit only.

I've never programmed for a VM environment, or for any flavor of Linux other then Linux emulators that run under Windows.

Post to thread

Message boards : News : do you have any experience with running a VM based application in BOINC?

//