By the way: things we'd need a comment on:

1. do PCs with multiple GPUs work as expected?
2. does suspend/restart work as expected?

Hi Toni,
i got 2 of them.
The 1. was at 32% when i suspend it, the 2. startet.
I restarted the 1.WU, and when 1 suspend the 2.WU to continue the 1. it exit with an error.
Then the 2. (still was at 0.0%) startet from itself.

At 37% i suspend and restartet it, it also exit with an error.

You can find my WU's here

GTX 1660 Ti and Windows 10

I hope it helps you to improve the app.

Hey Toni,

The one I received errored out with only one 2080ti and no other cards in the system on windows:

http://www.gpugrid.net/result.php?resultid=21344094

http://www.gpugrid.net/workunit.php?wuid=16749264
This WU run concurrently with E@H fine (2x GTX1060's). Suspended it once with leave WU in memory and it restarted fine from where it left off. Same with another ACEMD 2.06 WU but on another machine with only one GTX1060.

By the way: things we'd need a comment on:

1. do PCs with multiple GPUs work as expected?
2. does suspend/restart work as expected?

I ran 2 of the units on my windows 10 machine. This machine has a gtx 980 ti, which was running long unit, while the rtx 2080 ti was running the new version of ACEMD v2.06 (cuda100)unit. When I let the test unit run from start to finish without interruption, it finishes successfully, but when I suspend it and then resume it, it will crash within a few seconds. GPU usage on this machine was 80% maximum, compared to 90% usage for the long run, which was running on the 980 ti.

http://www.gpugrid.net/results.php?hostid=263612&offset=0&show_names=1&state=0&appid=32

I've also had a test app have an error when suspended then resumed. Currently have one running. I'll let it go to see if it goes to completion.

The workunit seems to be using 1 full CPU core and 92% GPU Load.
My CPU is i7-4790 and GPU is GTX 1660.

This test WU was suspended twice (once using Suspend, once using Suspend GPU in BOINC Manager) and successfully restarted and completed.

http://www.gpugrid.net/result.php?resultid=21354832

This test WU was suspended twice (once using Suspend, once using Suspend GPU in BOINC Manager) and successfully restarted and completed.

http://www.gpugrid.net/result.php?resultid=21354832

This test WU was suspended twice (once using Suspend, once using Suspend GPU in BOINC Manager) and successfully restarted and completed.

http://www.gpugrid.net/result.php?resultid=21354832

You're running linux with a GTX1080 card, while I am running windows with a RTX card. This is either a OS problem or a card type problem. To determine what is the problem we need to run these WU's on a non RTX card with windows and/or RTX card on linux.

9/7/2019 9:44:40 AM | GPUGRID | task a70-TONI_TESTDHFR206b-9-30-RND0994_0

This task assigned to i7 Windows 10 and RTX 2080: Processed about 1:00 minute, suspended for 30 seconds, resumed and the task immediately failed.

Running OK on 2x GPU system:
https://www.gpugrid.net/results.php?hostid=475308

Results show a -device 0 or -device 1.

Just received another test task. Decided to check the suspend/resume. Computation error on resume still. GTX1660

I continue to have no luck getting any of these new test tasks.

TONI:

The GPUGrid configuration (below)is set specifically to accommodate my i7, Windows 10 with RTX 2080. I momentarily selected both short and long runs ACEMD tasks and two immediately in sequence failed.
Do you wish us to continue the Pause-then-Resume on the ACEMD3 and other special test tasks for the RTX cards.

My three other machines with Windows and GTX 750ti and 1060s set idle as far as GPUGrid is concerned.

ACEMD short runs (2-3 hours on fastest card): no
ACEMD long runs (8-12 hours on fastest GPU): no
ACEMD3: yes
Quantum Chemistry (CPU): no
Quantum Chemistry (CPU, beta): no
Python Runtime: no
If no work for selected applications is available, accept work from other applications?no

The GPUGrid configuration (below)is set specifically to accommodate my i7, Windows 10 with RTX 2080. I momentarily selected both short and long runs ACEMD tasks and two immediately in sequence failed.

My three other machines with Windows and GTX 750ti and 1060s set idle as far as GPUGrid is concerned.

Since I have been unable to get any of these new acemd3 tasks, is it valid to say that only the Windows hosts are having issues? And that the Linux hosts continue to not have any issues with the new app or tasks? I've only seen one post from a Linux user saying they had no issues.

I was hoping to test for myself the new apps and higher rewarding tasks. I had no issues with the previous beta and tasks back in July. No such luck for the new apps and tasks in retrieving either so far.

And that the Linux hosts continue to not have any issues with the new app or tasks? I've only seen one post from a Linux user saying they had no issues.

Since I have been unable to get any of these new acemd3 tasks, is it valid to say that only the Windows hosts are having issues?

My three other machines with Windows and GTX 750ti and 1060s set idle as far as GPUGrid is concerned.

When I access my GpuGrid account and select Preferences and subsequently GpuGrid Preferences, whatever I select as to applications to run has always applied to all of the four computers I have attached to GpuGrid.

By the way: things we'd need a comment on:

2. does suspend/restart work as expected?

http://gpugrid.net/result.php?resultid=21378124

Linux machine with a GeForce GTX 1050 Ti (ran GPUGrid task) and GeForce GTX 660 (running another project). Survived two suspends, including a machine reboot, and finished happily.

Seems that others weren't so lucky with the same work unit though.

I lucked out and checked in just when a new tranche of acemd3 WUs popped up. My 2080 Ti caught two sets of two WUs and they ran fine.
My 2080 Ti has no problem running 4 einstein WUs but I've yet to get 4 WUs at the same time to test this for acemd3. Is there a limitation set on the server side???
____________

Sigh . . . . still have never caught a single one of the new tasks or applications.

Keith it took me a while before I caught my first. Are you sure you have acemd3 checked in preferences and short & long unchecked???
____________

Keith it took me a while before I caught my first. Are you sure you have acemd3 checked in preferences and short & long unchecked???

I set my three LINUX hosts to "no new work". So, that Keith can pick one LINUX WU up;-)

Be patient: It seems to me, that we are nearing to production status with the new app.

I set my three LINUX hosts to "no new work". So, that Keith can pick one LINUX WU up;-)

I set my three LINUX hosts to "no new work". So, that Keith can pick one LINUX WU up;-)

Be patient: It seems to me, that we are nearing to production status with the new app.

Just realized I successfully processed three of the "new" tasks on a Linux system with one 1660ti and five 1060. They all completed successfully. I didn't realize they were running so I failed to do a stop start to test suspend.

I also noticed that the GPU was not identified so I don't know if they all ran on gpu0 or any of the other 5 on this mining system using risers for all cards. One of them was faster so I suspect that was on the 1660ti.

http://www.gpugrid.org/results.php?hostid=509037


Would be interesting to compare to other system that have full 16x bandwidth instead of my 1x.

I believe other Linux users have already tested the new acemd3 app for stops, suspends and restarts with no issues. They processed through to completion, even on different cards I believe.

That is what the Windows app needs to achieve.

One of them was faster so I suspect that was on the 1660ti.

http://www.gpugrid.org/results.php?hostid=509037

Would be interesting to compare to other system that have full 16x bandwidth instead of my 1x.

One of them was faster so I suspect that was on the 1660ti.

http://www.gpugrid.org/results.php?hostid=509037

Would be interesting to compare to other system that have full 16x bandwidth instead of my 1x.

For comparison, another Volunteer crunched this task on Linux host with GTX1660ti
http://www.gpugrid.net/result.php?resultid=21381326

Hi all, noob here :-)

I have three 2080ti cards (on Windows) and I'm trying to get work units but Boinc keeps saying there aren't any available since I started yesterday:

roryd, I just go ahead and check the acemd project as well. The test WUs come in small packs so it's catch as catch can. Keep an eye on the Server Status page.
____________

roryd, I just go ahead and check the acemd project as well. The test WUs come in small packs so it's catch as catch can. Keep an eye on the Server Status page.

There is a loss in performance of %16 due to x1 but on the other hand, Windows with 1070Ti and a full x16 is slightly slower than the 1660Ti hanging on a 1x riser on Ubuntu! Both of my systems have swan_sync enabled and both run CUDA 10.0 Not sure about the other user.

Sorry, my bad. I'm talking about 1080 Ti's and you're running 2080 Ti's. No, acemd does not work for Turing GPUs.

I get confused as my single 2080 Ti is on a Linux computer.
____________

Received a TEST work unit a43-TONI_TESTDHFR207c-23-30-RND4156_0 on a Win10 Host with GTX1060 GPU.

Applied the following test:
Let work unit run for 11 minutes 13 seconds
suspended for 1 minute 20 seconds (approx)
Resumed work unit.

Results:
Work unit had computational error several seconds after resuming

Observations:
Work unit predicted a run time of 36 minutes. This is an improvement on Work unit a89-TONI_TESTDHFR206b-23-30-RND6008_0 , which had a run time of 66 minutes. Speed issues seems to be improved.
ACEMD3 task and Wrapper task disappeared from Task Manager after suspending task.
After resumption / failure, the run time reverted to 2 minutes 12 seconds.
STDerr Output time line reflects the full run time of 11 minutes, but Run Time summary only reflects 2 minutes 12 seconds.
nvidia-smi reported 78% GPU utilization which is inline with CUDA80 tasks on this host.
nvidia-smi reported similar Power usage as CUDA80 tasks on this host.

Link to Work unit here:
http://gpugrid.net/result.php?resultid=21396885

Received another TEST work unit a6-TONI_TESTDHFR207-2-3-RND1704

Same testing method as last post, this time allowed work unit to run 40 minutes 37 seconds before suspending. (54% complete)
Task failed after resuming 1 minute later.

The run time may not have improved as indicated in last post. After 40 minutes 37 seconds task was 54% completed. So Windows 10 tasks still seem to have a speed issue compared to Linus ACEMD3 tasks.

Additional I did notice the ACEMD3 task and Wrapper task did reappear in Task Manager for a few seconds before the task failed.

All other observations consistent with last post.

Failed task here:
http://gpugrid.net/result.php?resultid=21397083

Failed task here:
http://gpugrid.net/result.php?resultid=21397083

what caught my eye:
in line 8 of the stderr it says "Detected memory leaks!" - whatever this means.

My following W10 computer, GTX1050TI graphics card:
https://www.gpugrid.net/show_host_detail.php?hostid=105442
Got an ACEMD3 V2.07 test WU:
https://www.gpugrid.net/result.php?resultid=21400000
It was processed till the end, no pauses, and then errored out with indication "195 (0xc3) EXIT_CHILD_FAILED".
The same WU but V2.06 was processed successfully by a second computer:
WU: https://www.gpugrid.net/result.php?resultid=21400083
Computer: https://www.gpugrid.net/show_host_detail.php?hostid=459450
Something is still to be polished in V2.07 application code and/or scheduler...

I also miss previously available information in old ACEMD WUs about graphics card model, clocks, temperatures reached...
Perhaps it is not possible in new WUs due to Wrapper's philosophy (?)

e1s20_ubiquitin_50ns_3-ADRIA_FOLDUBQ_BANDIT_crystal_ss_contacts_50_ubiquitin_2-0-2-RND1315.
This task had downloaded and failed without my immediate knowledge as I was doing some routine computer updating/restarting. Failure occurred at 52.31 seconds which is of course irrelevant. Machine: I7 Windows10 RTX2080.

Do I understand that Toni still wants these failure reports?

I lost a pair of those "new" tasks

http://www.gpugrid.org/results.php?hostid=467730&offset=0&show_names=0&state=5&appid=

I had to reboot to install to fix a problem with an app. I did stop the Boinc client before rebooting but the two "new" tasks did not survive the reboot.

Looking at errors, my only other errors on this system were almost a year ago.

Just had my first failure on a restarted CUDA100 task that obeyed the set 60 minute run per project setting. Restarted on a different device and failed.

Stderr output
<core_client_version>7.16.2</core_client_version>
<![CDATA[
<message>
process exited with code 195 (0xc3, -61)</message>
<stderr_txt>
18:14:25 (25157): wrapper (7.7.26016): starting
18:14:25 (25157): wrapper (7.7.26016): starting
18:14:25 (25157): wrapper: running acemd3 (--boinc input --device 0)
21:09:51 (23108): wrapper (7.7.26016): starting
21:09:51 (23108): wrapper (7.7.26016): starting
21:09:51 (23108): wrapper: running acemd3 (--boinc input --device 1)
ERROR: /home/user/conda/conda-bld/acemd3_1566914012210/work/src/mdsim/context.cpp line 324: Cannot use a restart file on a different device!
21:09:55 (23108): acemd3 exited; CPU time 3.826201
21:09:55 (23108): app exit status: 0x9e
21:09:55 (23108): called boinc_finish(195)

</stderr_txt>
]]>

So this has a failure in common with the Windows wrapper app.

https://www.gpugrid.net/result.php?resultid=21408545

I see the same as others on suspend.

http://www.gpugrid.net/result.php?resultid=21408995

These WU's don't stress my Titan V much the load is about 75% and power 100W. My 10xx GPU's are another 10% load and double power draw.

I noticed that the new ACEMD3 Windows app v2.06 does not update the boinc_task_state.xml file in the slot directory.
It maybe related to the checkpoint + "resuming does not work" issue.
BTW I don't know why this host received the v2.06, instead of the more recent v2.07. (NVIDIA GeForce GTX 1080 Ti (4095MB) driver: 436.15)

I restarted my machine and the WU crashed.

e1s29_ubiquitin_50ns_5-ADRIA_FOLDUBQ_BANDIT_crystal_ss_contacts_50_ubiquitin_1-1-2-RND8951_2

Stderr output
<core_client_version>7.14.2</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code 195 (0xc3)</message>
<stderr_txt>
19:49:47 (10992): wrapper (7.9.26016): starting
19:49:47 (10992): wrapper: running acemd3.exe (--boinc input --device 0)
Detected memory leaks!
Dumping objects ->
..\api\boinc_api.cpp(309) : {1755} normal block at 0x000001D0701589B0, 8 bytes long.
Data: < p > 00 00 11 70 D0 01 00 00
..\lib\diagnostics_win.cpp(417) : {202} normal block at 0x000001D07015C010, 1080 bytes long.
Data: <@ > 40 02 00 00 CD CD CD CD E0 01 00 00 00 00 00 00
Object dump complete.
22:43:23 (11696): wrapper (7.9.26016): starting
22:43:23 (11696): wrapper: running acemd3.exe (--boinc input --device 0)
# Engine failed: The periodic box size has decreased to less than twice the nonbonded cutoff.
22:43:27 (11696): acemd3.exe exited; CPU time 0.000000
22:43:27 (11696): app exit status: 0x1
22:43:27 (11696): called boinc_finish(195)
0 bytes in 0 Free Blocks.
506 bytes in 8 Normal Blocks.
1144 bytes in 1 CRT Blocks.
0 bytes in 0 Ignore Blocks.
0 bytes in 0 Client Blocks.
Largest number used: 0 bytes.
Total allocations: 141102 bytes.
Dumping objects ->
{1814} normal block at 0x0000016B06965620, 48 bytes long.
Data: <ACEMD_PLUGIN_DIR> 41 43 45 4D 44 5F 50 4C 55 47 49 4E 5F 44 49 52
{1803} normal block at 0x0000016B069657E0, 48 bytes long.
Data: <HOME=D:\ProgramD> 48 4F 4D 45 3D 44 3A 5C 50 72 6F 67 72 61 6D 44
{1792} normal block at 0x0000016B06965BD0, 48 bytes long.
Data: <TMP=D:\ProgramDa> 54 4D 50 3D 44 3A 5C 50 72 6F 67 72 61 6D 44 61
{1781} normal block at 0x0000016B06965690, 48 bytes long.
Data: <TEMP=D:\ProgramD> 54 45 4D 50 3D 44 3A 5C 50 72 6F 67 72 61 6D 44
{1770} normal block at 0x0000016B06965150, 48 bytes long.
Data: <TMPDIR=D:\Progra> 54 4D 50 44 49 52 3D 44 3A 5C 50 72 6F 67 72 61
{1759} normal block at 0x0000016B069460C0, 141 bytes long.
Data: <<project_prefere> 3C 70 72 6F 6A 65 63 74 5F 70 72 65 66 65 72 65
..\api\boinc_api.cpp(309) : {1756} normal block at 0x0000016B06966A10, 8 bytes long.
Data: < k > 00 00 93 06 6B 01 00 00
{981} normal block at 0x0000016B06944D40, 141 bytes long.
Data: <<project_prefere> 3C 70 72 6F 6A 65 63 74 5F 70 72 65 66 65 72 65
{204} normal block at 0x0000016B069661F0, 8 bytes long.
Data: < k > 10 90 96 06 6B 01 00 00
{197} normal block at 0x0000016B06965460, 48 bytes long.
Data: <--boinc input --> 2D 2D 62 6F 69 6E 63 20 69 6E 70 75 74 20 2D 2D
{196} normal block at 0x0000016B06966BA0, 16 bytes long.
Data: < k > A8 C4 94 06 6B 01 00 00 00 00 00 00 00 00 00 00
{195} normal block at 0x0000016B069662E0, 16 bytes long.
Data: < k > 80 C4 94 06 6B 01 00 00 00 00 00 00 00 00 00 00
{194} normal block at 0x0000016B06966BF0, 16 bytes long.
Data: <X k > 58 C4 94 06 6B 01 00 00 00 00 00 00 00 00 00 00
{193} normal block at 0x0000016B06966010, 16 bytes long.
Data: <0 k > 30 C4 94 06 6B 01 00 00 00 00 00 00 00 00 00 00
{192} normal block at 0x0000016B06966290, 16 bytes long.
Data: < k > 08 C4 94 06 6B 01 00 00 00 00 00 00 00 00 00 00
{191} normal block at 0x0000016B06966F60, 16 bytes long.
Data: < k > E0 C3 94 06 6B 01 00 00 00 00 00 00 00 00 00 00
{190} normal block at 0x0000016B06965540, 48 bytes long.
Data: <ComSpec=C:\Windo> 43 6F 6D 53 70 65 63 3D 43 3A 5C 57 69 6E 64 6F
{189} normal block at 0x0000016B06966150, 16 bytes long.
Data: <@O k > 40 4F 96 06 6B 01 00 00 00 00 00 00 00 00 00 00
{188} normal block at 0x0000016B0695A5D0, 32 bytes long.
Data: <SystemRoot=C:\Wi> 53 79 73 74 65 6D 52 6F 6F 74 3D 43 3A 5C 57 69
{187} normal block at 0x0000016B06966420, 16 bytes long.
Data: < O k > 18 4F 96 06 6B 01 00 00 00 00 00 00 00 00 00 00
{185} normal block at 0x0000016B069666A0, 16 bytes long.
Data: < N k > F0 4E 96 06 6B 01 00 00 00 00 00 00 00 00 00 00
{184} normal block at 0x0000016B06966650, 16 bytes long.
Data: < N k > C8 4E 96 06 6B 01 00 00 00 00 00 00 00 00 00 00
{183} normal block at 0x0000016B06966C40, 16 bytes long.
Data: < N k > A0 4E 96 06 6B 01 00 00 00 00 00 00 00 00 00 00
{182} normal block at 0x0000016B06966DD0, 16 bytes long.
Data: <xN k > 78 4E 96 06 6B 01 00 00 00 00 00 00 00 00 00 00
{181} normal block at 0x0000016B06966510, 16 bytes long.
Data: <PN k > 50 4E 96 06 6B 01 00 00 00 00 00 00 00 00 00 00
{180} normal block at 0x0000016B06964E50, 280 bytes long.
Data: < e k PQ k > 10 65 96 06 6B 01 00 00 50 51 96 06 6B 01 00 00
{179} normal block at 0x0000016B06966100, 16 bytes long.
Data: < k > C0 C3 94 06 6B 01 00 00 00 00 00 00 00 00 00 00
{178} normal block at 0x0000016B069661A0, 16 bytes long.
Data: < k > 98 C3 94 06 6B 01 00 00 00 00 00 00 00 00 00 00
{177} normal block at 0x0000016B06966560, 16 bytes long.
Data: <p k > 70 C3 94 06 6B 01 00 00 00 00 00 00 00 00 00 00
{176} normal block at 0x0000016B0694C370, 496 bytes long.
Data: <`e k acemd3.e> 60 65 96 06 6B 01 00 00 61 63 65 6D 64 33 2E 65
{65} normal block at 0x0000016B069590E0, 16 bytes long.
Data: < > 80 EA 11 A0 F6 7F 00 00 00 00 00 00 00 00 00 00
{64} normal block at 0x0000016B06959950, 16 bytes long.
Data: <@ > 40 E9 11 A0 F6 7F 00 00 00 00 00 00 00 00 00 00
{63} normal block at 0x0000016B06959770, 16 bytes long.
Data: < W > F8 57 0E A0 F6 7F 00 00 00 00 00 00 00 00 00 00
{62} normal block at 0x0000016B06959040, 16 bytes long.
Data: < W > D8 57 0E A0 F6 7F 00 00 00 00 00 00 00 00 00 00
{61} normal block at 0x0000016B06959720, 16 bytes long.
Data: <P > 50 04 0E A0 F6 7F 00 00 00 00 00 00 00 00 00 00
{60} normal block at 0x0000016B06959680, 16 bytes long.
Data: <0 > 30 04 0E A0 F6 7F 00 00 00 00 00 00 00 00 00 00
{59} normal block at 0x0000016B069595E0, 16 bytes long.
Data: < > E0 02 0E A0 F6 7F 00 00 00 00 00 00 00 00 00 00
{58} normal block at 0x0000016B06958FF0, 16 bytes long.
Data: < > 10 04 0E A0 F6 7F 00 00 00 00 00 00 00 00 00 00
{57} normal block at 0x0000016B06958F00, 16 bytes long.
Data: <p > 70 04 0E A0 F6 7F 00 00 00 00 00 00 00 00 00 00
{56} normal block at 0x0000016B06959590, 16 bytes long.
Data: < > 18 C0 0C A0 F6 7F 00 00 00 00 00 00 00 00 00 00
Object dump complete.

</stderr_txt>
]]>

No issues here with suspending and resuming tasks under Linux. Just suspended a WU and it resumed on the other GPU in that box without issue (both GPUs are RTX 2080's).

No issues here with suspending and resuming tasks under Linux. Just suspended a WU and it resumed on the other GPU in that box without issue (both GPUs are RTX 2080's).

FWIW, my single machine with two GPUs will successfully process CUDA 101 tasks, but fail on CUDA 100 tasks. My other three machines with a single GPU will successfully process both CUDA 101 and CUDA 100 tasks.
____________
Reno, NV
Team: SETI.USA

No issues here with suspending and resuming tasks under Linux. Just suspended a WU and it resumed on the other GPU in that box without issue (both GPUs are RTX 2080's).

Curious. I wonder if my Linux failure was because the paused task did not start back up on the same type of card.

I have a request for help from Windows users. Does anyone want to try a development branch of the client that may be able to handle the pause/suspend issues on the acemd3 wrapper apps?

I was browsing through the latest commits and and came upon PR#3307 which has the tantalizing description of

Description of the Change
On Windows, CreateProcess() is used to launch tasks, but this on its own does not handle child processes; if the parent task process exits, the workunit will be terminated. If <wait_for_children> is set in the job file, attach the task process to a job object instead, which can then be monitored to determine when all child processes are finished.

Alternate Designs

Release Notes

Add <wait_for_children> option for tasks in job.xml

FWIW, my single machine with two GPUs will successfully process CUDA 101 tasks, but fail on CUDA 100 tasks. My other three machines with a single GPU will successfully process both CUDA 101 and CUDA 100 tasks.

The one host that is getting the dominant amount of new acemd3 work just so happens to have three identical EVGA GTX 1070 Ti Black Edition cards and the tasks can apparently restart and run on any one of them after being switched off by the "switch between projects" standard 60 minute delimiter.

I have a request


You can download the latest AppVeyor artifact here for the client.

https://ci.appveyor.com/api/buildjobs/y4gd2lvbjjwoa54l/artifacts/deploy%2Fwin-client%2Fwin-client_PR3307_2019-09-26_8665946a.7z

There are two different gt1030's. One is significantly slower than the other else they are identical. The newer versions are crippled. I was wondering if the pair you have together are matched. Just a guess as that could cause unexpected timing values if the apps simply checks the name and does not bother to recalculate parameters.

It is still from the master branch which is the development version 7.15.0.


Or at least it still has the versioning number from the master branch. It may have more commits from further upstream too. If the version.h and version.log files aren't updated, the compile will still show whatever the version in those files are set.

But it has the commit I referenced in it with a fix for wrapper apps.

I have a request for help from Windows users. Does anyone want to try a development branch of the client that may be able to handle the pause/suspend issues on the acemd3 wrapper apps?

I was browsing through the latest commits and and came upon PR#3307 which has the tantalizing description of

Description of the Change
On Windows, CreateProcess() is used to launch tasks, but this on its own does not handle child processes; if the parent task process exits, the workunit will be terminated. If <wait_for_children> is set in the job file, attach the task process to a job object instead, which can then be monitored to determine when all child processes are finished.

Alternate Designs

Release Notes

Add <wait_for_children> option for tasks in job.xml

This sounds like it may address some of the error messages I see in stderr.txt when a wrapper app is suspended or paused. And why Toni has asked whether the wrapper app and the child process acemd3 app are still in the Task Manager list.

You can download the latest AppVeyor artifact here for the client.

https://ci.appveyor.com/api/buildjobs/y4gd2lvbjjwoa54l/artifacts/deploy%2Fwin-client%2Fwin-client_PR3307_2019-09-26_8665946a.7z

Hi I'm new on this forum.
Here is what I've got with my 1050ti on linux x64 :
curent task : ADRIA_FOLDUBQ_BANDIT_ss_contacts_50_ubiquitin_4-0-2
resources : 0.909 CPUs + 1 NVIDIA GPU
task size : 5000000 GFLOPs
elapsed time : 08:54:01
remaining time : 09:09:51
progress : 13,800 %

14% done after 50% elapsed time ???

Here is what I've got with my 1050ti on linux x64 :
remaining time : 09:09:51
progress : 13,800 %
14% done after 50% elapsed time ???

I'm a little worried by that. The changes in PR #3307 were made in the wrapper app itself (only). You could indeed download the win-apps bundle from appveyor and extract wrapper_26014_windows_x86_64.exe, but it would be hard to deploy if Toni is issuing an earlier version from the server.

If the client downloaded from that link has improvements, they'll come from the cumulative set of changes made both before and after the 7.16 branch was split. We urgently need to work out which the beneficial change was, and whether it happened before or after the fork. If it was made later, it needs to be cherrypicked into the new release.

Are you saying that the wrapper app needs to be updated in the server code? Like in the new 1.20 server release?

Hi I'm new on this forum.
Here is what I've got with my 1050ti on linux x64 :
curent task : ADRIA_FOLDUBQ_BANDIT_ss_contacts_50_ubiquitin_4-0-2
resources : 0.909 CPUs + 1 NVIDIA GPU
task size : 5000000 GFLOPs
elapsed time : 08:54:01
remaining time : 09:09:51
progress : 13,800 %

14% done after 50% elapsed time ???

Given that the Appveyor version I downloaded from your link this morning was 26014, it looks like Toni has possibly been updating his own local copy along the way, and getting ahead of BOINC Central. If so, I hope he pushes back any useful changes to GitHub when he's got it all working.

But that's all just guesswork. Only Toni could tell you for certain.

The calculations for estimated remaining time tend to give rather inaccurate values under new versions until at least ten other tasks with the new version have run on the same computer.

9/29/2019 9:55:41 AM | GPUGRID | Computation for task e16s9_e14s4p0f17-ADRIA_FOLDUBQ_BANDIT_crystal_ss_contacts_50_ubiquitin_0-0-2-RND4379_1 finished

This 2.07 task failed 8.75 seconds after startup following a management activity on my part. I had NOT suspended boinc manager activity before the shutdown.

The machine is I7, W10, RTX2080.

Again frustrating that at this point the problem has not been solved.

Again TONI do you want these individual reports?

no task failed on linux.
1st unit : i7 with 1x 1050ti (cuda80 tasks)
2nd unit : i5 with 2x 1060 (cuda100 tasks)

Dears, sorry for the slow progress but I determined (at least) a restart problem, and it is not related to the wrapper. It is Windows-only, CUDA 10 only, as far as I can tell from your reports, and manifests itself with the

"The periodic box size has decreased to less than twice the nonbonded cutoff."

message.

Unfortunately the root cause is hard to identify (may be external to our code).

I have compiled the wrapper myself (the binaries on the boinc page are old and had one important bug in variable substitution), but for now the failures seem unrelated.

It's a bit frustrating because everything else seems to work nicely.

No issues here with suspending and resuming tasks under Linux. Just suspended a WU and it resumed on the other GPU in that box without issue (both GPUs are RTX 2080's).

Curious. I wonder if my Linux failure was because the paused task did not start back up on the same type of card.

Edit - I forgot about the problem reported by Keith Myers involving suspend/resume on different types of cards. I guess this will need to be fixed before it can be released.

Any chance the Linux app could be released now, since the Linux community has been without steady work for months and the Linux app seems to be working fine? Please, please please.

There is not enough work even for the Windows based hosts in the past few months. There would be much more complaints for the lack of work if the Linux community could also crunch them. BTW I am in both groups, but I prefer Linux for the higher performance due to the lack of WDDM.

Complaints? Have they ever stopped?

My GPU feels extremely neglected 😪

If you want something to crunch, folding@home always has work and is always looking for more volunteers

If you want something to crunch, folding@home always has work and is always looking for more volunteers

I have decided to not restart or fiddle around with my machine. Let us see if it finishes successfully.
If I may say so, according to Afterburner my GPU is running four degrees centigrade hotter then the old WU or normal. As hot as Einstien@Home which is my backup when GPU Grid becomes lazy. Resources set at zero for Einstien.

Lost it. Power failure.

Dears, sorry for the slow progress but I determined (at least) a restart problem, and it is not related to the wrapper. It is Windows-only, CUDA 10 only, as far as I can tell from your reports, and manifests itself with the

"The periodic box size has decreased to less than twice the nonbonded cutoff."

message.

Unfortunately the root cause is hard to identify (may be external to our code).

I have compiled the wrapper myself (the binaries on the boinc page are old and had one important bug in variable substitution), but for now the failures seem unrelated.

It's a bit frustrating because everything else seems to work nicely.

Hi,
there is not only CUDA 10.
for example:
http://www.gpugrid.net/result.php?resultid=21425993
http://www.gpugrid.net/result.php?resultid=21425617
http://www.gpugrid.net/result.php?resultid=21425545
#SWAN: FATAL: cannot find image for module [.nonbonded.cu.] for device version 750

Stderr Ausgabe
<core_client_version>7.14.2</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code -59 (0xffffffc5)</message>
<stderr_txt>
# GPU [GeForce RTX 2080 Ti] Platform [Windows] Rev [3212] VERSION [80]
# SWAN Device 0 :
# Name : GeForce RTX 2080 Ti
# ECC : Disabled
# Global mem : 11264MB
# Capability : 7.5
# PCI ID : 0000:01:00.0
# Device clock : 1755MHz
# Memory clock : 7000MHz
# Memory width : 352bit
# Driver version : r436_45 : 43648
#SWAN: FATAL: cannot find image for module [.nonbonded.cu.] for device version 750

</stderr_txt>
]]>

Hi,
there is not only CUDA 10.
for example:
http://www.gpugrid.net/result.php?resultid=21425993
http://www.gpugrid.net/result.php?resultid=21425617
http://www.gpugrid.net/result.php?resultid=21425545
#SWAN: FATAL: cannot find image for module [.nonbonded.cu.] for device version 750

Stderr Ausgabe
<core_client_version>7.14.2</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code -59 (0xffffffc5)</message>
<stderr_txt>
# GPU [GeForce RTX 2080 Ti] Platform [Windows] Rev [3212] VERSION [80]
# SWAN Device 0 :
# Name : GeForce RTX 2080 Ti
# ECC : Disabled
# Global mem : 11264MB
# Capability : 7.5
# PCI ID : 0000:01:00.0
# Device clock : 1755MHz
# Memory clock : 7000MHz
# Memory width : 352bit
# Driver version : r436_45 : 43648
#SWAN: FATAL: cannot find image for module [.nonbonded.cu.] for device version 750

</stderr_txt>
]]>

Turing GPU cards are only able to do TEST Work Units at the moment.
You will need to change your GPUGRID settings to ensure only TEST Work Units are accepted for your Turing GPU.
The above errors occur for ACEMD2 Work units on Turing based cards.

7.2.42 is really old (but latest on berkeley download). very likely the client is estimating wrong in addition to mis-identifying the cpu. apt-get under ubuntu 18.04 got me version 7.16.1 boinc

He must have installed the ppa. The Ubuntu 18.04 distro only has BOINC 7.9.3.
Or he went to the source and compiled it on his own.

Turing GPU cards are only able to do TEST Work Units at the moment.
You will need to change your GPUGRID settings to ensure only TEST Work Units are accepted for your Turing GPU.
The above errors occur for ACEMD2 Work units on Turing based cards.

How is it the other way round? Will the TEST Work Units work with cards prior Turing?

how did you get 7.16?

Hi,
this is my last error on Windows after a suspend/resume action.

http://www.gpugrid.net/result.php?resultid=21426852

I hope it help.
K.
____________
Dreams do not always come true. But not because they are too big or impossible. Why did we stop believing.
(Martin Luther King)

how did you get 7.16?

https://launchpad.net/~costamagnagianfranco/+archive/ubuntu/boinc

Thank you for the expert advice. So far 7.16.3 has worked on three other Ubuntu machines and one Win7. I did get a lot of extra downloads on WCG that I had never seen before, but expect that is a problem at their end(?). That is mainly because I have seen their "settings" reset every few months, and don't entirely trust their servers on that. Also, I just did a manual update of BOINC with no more extraneous work units downloaded, so it seems to be OK now.

When will there be some WUs that will run without erroring out on Windows machines? And not suck up an entire CPU in addition to the GPU?

____________

Hi,
this is my last error on Windows after a suspend/resume action.

http://www.gpugrid.net/result.php?resultid=21426852

I hope it help.
K.

7.2.42 is really old (but latest on berkeley download). very likely the client is estimating wrong in addition to mis-identifying the cpu. apt-get under ubuntu 18.04 got me version 7.16.1 boinc

how did you get 7.16? I have same version of ubuntu but in repos see available only 7.9.3

When will there be some WUs that will run without erroring out on Windows machines? And not suck up an entire CPU in addition to the GPU?

They will always need their own CPU.

Some config changes :
- i5 have just 1 GTX1060 now (cuda100)
- replaced GTX1050ti by GTX1060 on i7 (cuda80)
current ADRIA's resumed task fail after I changed GPU.

Now all works perfectly even with multiple suspend/resume.

GTX1060 is definitely more much faster than GTX1050ti. (7h vs 95h for same task)