"snapshot" repackaging (was Re: [slicer-users] slicer DMRI Extensions)

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

"snapshot" repackaging (was Re: [slicer-users] slicer DMRI Extensions)

Isaiah Norton-2
Hi Andras,

This will not be a private build: it will be 100% binary compatible because it will be exactly Slicer.exe and libraries from the nightly, just repacked up with some additional extensions. I really do mean *repackaging* -- there will be no compiler running at all [1]. Very similar to Steve's link (thanks for the reminder) but much more limited in scope and running automatically on CI.

If/when general snapshots come online, we'll discontinue. This is a minimal experiment to improve the first-order problem of user install reliability within the scope of systems and issues I can control.

Isaiah

[1] well, I suppose technically NSIS is kind of a pascal compiler.


On Fri, Mar 17, 2017 at 10:52 AM, Andras Lasso <[hidden email]> wrote:

Would it work for you if we picked a nightly version once in every 1-2 months (that has no known major issues) and build all extensions for that every night?

 

Stable builds do almost exactly this except:

  • Created yearly instead of monthly
  • Download is very slow

 

Shouldn't we try to improve stable builds instead of building a limited build infrastructure in parallel? I'm afraid introducing additional build infrastructures would be more work overall, we would loose download stats, and confuse users. Or at least coordinate these private builds: make them binary compatible, collect download stats in one location, show all download links on the official Slicer download page, etc.

 

Andras

 

From: [hidden email]
Sent: Friday, March 17, 2017 10:23 AM
To: [hidden email]
Cc: [hidden email]; [hidden email]
Subject: RE: [slicer-users] slicer DMRI Extensions

 

It make sense then to choose a version that corresponds to a nightly build that the factory machines build extensions for.

 

By the way, building all the extensions is really simple and takes about the same time as building Slicer. The difficulty is that you have to set up the build on all platforms

 

Andras

 

From: [hidden email]
Sent: Thursday, March 16, 2017 6:08 PM
To: [hidden email]
Cc: [hidden email]; [hidden email]
Subject: Re: [slicer-users] slicer DMRI Extensions

 

Hi Andras,

Right now I'm aiming for something very simple: to repackage a known-good nightly build with our extensions installed, and uploaded as a GitHub release. I'm not planning to set up a separate build system. Doing repackaging-only should run within the time limits of the free versions of Appveyor (Windows) and Travis (Linux,Mac).

Would you make all other extensions available for these monthly snapshots? 

I'd like to get this going for SlicerDMRI before committing/commenting, but maybe opt-in. I definitely don't want to poll the entire extension index because there is a lot of unmaintained stuff and it will take too long.

Could you build a barebone installer, too, that doesn't contain any extensions?

The corresponding nightly installer for the given snapshot date would be the barebones installer.

Would the snapshot be a custom application (unique name, settings, icon, etc.)?

No. My goal is just a "nightly plus" from a known-good build. I don't want (nor have resources) to duplicate the existing build infrastructure.

Isaiah


On Thu, Mar 16, 2017 at 5:04 PM, Andras Lasso <[hidden email]> wrote:

Isaiah, having a monthly snapshot wpuld be great. Would you make all other extensions available for these monthly snapshots? Could you build a barebone installer, too, that doesn't contain any extensions? Would the snapshot be a custom application (unique name, settings, icon, etc.)?

 

Andras

 

From: [hidden email]
Sent: Thursday, March 16, 2017 4:48 PM
To: [hidden email]
Cc: [hidden email]
Subject: Re: [slicer-users] slicer DMRI Extensions

 

Hi Jeff,

Thank you for your patience, I apologize for these problems. Going forward we will be creating "snapshot" packages with SlicerDMRI pre-installed, on a regular basis (~monthly).

We will link to these snapshots as the recommended installation method on our website. The extension will still be available in the nightly too, but in light of various extension download issues [1], we are going to experiment with snapshots as a better way to create a smooth installation process.

To that end: I have created an initial snapshot against the Slicer Linux nightly from 3/12, available to download here:


I tested this on an Ubuntu 14.04 virtualbox VM, and the seeding/display features work as expected (attached screenshot [2]).  I will script up the process and create similar packages for Mac and Windows shortly, and upload to the same location. I hope that this works correctly, but please let me know if not.

Regarding 4.6.2 on Linux, I was able to reproduce the Eigen/TractDisplay library installation problem issue in a clean VM. I will investigate and create a patch as soon as possible.

Best,
Isaiah


[2]  

Inline image 1





On Thu, Mar 16, 2017 at 2:12 PM, Jeff Stevenson <[hidden email]> wrote:
Hi isaiah, fyi when the eigen notice pops up the module for fiber tract display is missing. Not sure if that is a coincidence, but without the display module its not usable. Is there a manual workaround?

I tried the latest release and was unable to download any of the diffusion tool kit. Gave script error dialogue. No love.
Jeff


From: Isaiah Norton
Date: Wednesday, March 1, 2017 at 9:28 AM
To: Jeff Stevenson
Cc: SPL Slicer Users
Subject: Re: [slicer-users] slicer DMRI Extensions

Hi Jeff,

Thanks for the feedback!

Unfortunately, on Linux it looks like SlicerDMRI, UKF, and a number of other modules have empty package download files in the current nightly (tried SlicerRT, TCIABrowser, DSCMRIAnalysis, SlicerPathology, IntensitySegmenter). There are messages in the Slicer error log about empty archives, but regrettably the extension manager doesn't currently provide any prominent error message about this type of install failure.


btw also found the newer releases crash when extension manager is called when older versions are in the .config folder.
moving them out solves the problem with latest releases after feb 1, tho inconvenient.

Not sure about this one -- I believe each version should have its own config sub-folder, and at least on Mac any downloaded extensions are stored inside the version-specific app bundle.

The Eigen notice is present on all platforms but should not affect functionality of either the main DMRI modules or UKF, when they are otherwise installed correctly. I'll see if I can find a way to fix the message.

Best,
Isaiah


On Tue, Feb 21, 2017 at 2:03 PM, Jeff Stevenson <[hidden email]> wrote:
Hi slicer folks, been following the latest module releases of DMRI and UKFT . i tried to download the same dev version today 4.7.0-2017-02-18 r25712.
on my mac the module install fine. on my 14.04 linux workstation i get a dependency error

“the extension Eigen could not be found. the extension
may not work properly.”

tractography display seems not to be installed when this happens, so DMRI/UKFT build on linux still has some issues.

cheers jeff

btw also found the newer releases crash when extension manager is called when older versions are in the .config folder.
moving them out solves the problem with latest releases after feb 1, tho inconvenient.

_______________________________________________
slicer-users mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-users
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/4.3/FAQ





_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/Nightly/Developers/FAQ
Reply | Threaded
Open this post in threaded view
|

Re: "snapshot" repackaging (was Re: [slicer-users] slicer DMRI Extensions)

lasso2

Thanks for the clarification. I’ll try to get the release process simplified and have more frequent releases. For example, one possibility of simplification is to generate version-specific user documentation using Sphinx instead of cloning a large number wiki pages for each release.

 

Andras

 

From: Isaiah Norton [mailto:[hidden email]]
Sent: March 17, 2017 12:04
To: Andras Lasso <[hidden email]>
Cc: slicer-devel <[hidden email]>
Subject: "snapshot" repackaging (was Re: [slicer-users] slicer DMRI Extensions)

 

Hi Andras,

 

This will not be a private build: it will be 100% binary compatible because it will be exactly Slicer.exe and libraries from the nightly, just repacked up with some additional extensions. I really do mean *repackaging* -- there will be no compiler running at all [1]. Very similar to Steve's link (thanks for the reminder) but much more limited in scope and running automatically on CI.

 

If/when general snapshots come online, we'll discontinue. This is a minimal experiment to improve the first-order problem of user install reliability within the scope of systems and issues I can control.

 

Isaiah

 

[1] well, I suppose technically NSIS is kind of a pascal compiler.

 

 

On Fri, Mar 17, 2017 at 10:52 AM, Andras Lasso <[hidden email]> wrote:

Would it work for you if we picked a nightly version once in every 1-2 months (that has no known major issues) and build all extensions for that every night?

 

Stable builds do almost exactly this except:

  • Created yearly instead of monthly
  • Download is very slow

 

Shouldn't we try to improve stable builds instead of building a limited build infrastructure in parallel? I'm afraid introducing additional build infrastructures would be more work overall, we would loose download stats, and confuse users. Or at least coordinate these private builds: make them binary compatible, collect download stats in one location, show all download links on the official Slicer download page, etc.

 

Andras

 

From: [hidden email]
Sent: Friday, March 17, 2017 10:23 AM
To: [hidden email]
Cc: [hidden email]; [hidden email]
Subject: RE: [slicer-users] slicer DMRI Extensions

 

It make sense then to choose a version that corresponds to a nightly build that the factory machines build extensions for.

 

By the way, building all the extensions is really simple and takes about the same time as building Slicer. The difficulty is that you have to set up the build on all platforms

 

Andras

 

From: [hidden email]
Sent: Thursday, March 16, 2017 6:08 PM
To: [hidden email]
Cc: [hidden email]; [hidden email]
Subject: Re: [slicer-users] slicer DMRI Extensions

 

Hi Andras,

 

Right now I'm aiming for something very simple: to repackage a known-good nightly build with our extensions installed, and uploaded as a GitHub release. I'm not planning to set up a separate build system. Doing repackaging-only should run within the time limits of the free versions of Appveyor (Windows) and Travis (Linux,Mac).

 

Would you make all other extensions available for these monthly snapshots? 

 

I'd like to get this going for SlicerDMRI before committing/commenting, but maybe opt-in. I definitely don't want to poll the entire extension index because there is a lot of unmaintained stuff and it will take too long.

 

Could you build a barebone installer, too, that doesn't contain any extensions?

 

The corresponding nightly installer for the given snapshot date would be the barebones installer.

 

Would the snapshot be a custom application (unique name, settings, icon, etc.)?

 

No. My goal is just a "nightly plus" from a known-good build. I don't want (nor have resources) to duplicate the existing build infrastructure.

 

Isaiah

 

 

On Thu, Mar 16, 2017 at 5:04 PM, Andras Lasso <[hidden email]> wrote:

Isaiah, having a monthly snapshot wpuld be great. Would you make all other extensions available for these monthly snapshots? Could you build a barebone installer, too, that doesn't contain any extensions? Would the snapshot be a custom application (unique name, settings, icon, etc.)?

 

Andras

 

From: [hidden email]
Sent: Thursday, March 16, 2017 4:48 PM
To: [hidden email]
Cc: [hidden email]
Subject: Re: [slicer-users] slicer DMRI Extensions

 

Hi Jeff,

 

Thank you for your patience, I apologize for these problems. Going forward we will be creating "snapshot" packages with SlicerDMRI pre-installed, on a regular basis (~monthly).

 

We will link to these snapshots as the recommended installation method on our website. The extension will still be available in the nightly too, but in light of various extension download issues [1], we are going to experiment with snapshots as a better way to create a smooth installation process.

 

To that end: I have created an initial snapshot against the Slicer Linux nightly from 3/12, available to download here:

 

 

I tested this on an Ubuntu 14.04 virtualbox VM, and the seeding/display features work as expected (attached screenshot [2]).  I will script up the process and create similar packages for Mac and Windows shortly, and upload to the same location. I hope that this works correctly, but please let me know if not.

 

Regarding 4.6.2 on Linux, I was able to reproduce the Eigen/TractDisplay library installation problem issue in a clean VM. I will investigate and create a patch as soon as possible.

 

Best,

Isaiah

 

 

[2]  

 

Inline image 1

 

 

 

 

 

On Thu, Mar 16, 2017 at 2:12 PM, Jeff Stevenson <[hidden email]> wrote:

Hi isaiah, fyi when the eigen notice pops up the module for fiber tract display is missing. Not sure if that is a coincidence, but without the display module its not usable. Is there a manual workaround?

 

I tried the latest release and was unable to download any of the diffusion tool kit. Gave script error dialogue. No love.

Jeff

 

 

From: Isaiah Norton
Date: Wednesday, March 1, 2017 at 9:28 AM
To: Jeff Stevenson
Cc: SPL Slicer Users
Subject: Re: [slicer-users] slicer DMRI Extensions

 

Hi Jeff,

 

Thanks for the feedback!

 

Unfortunately, on Linux it looks like SlicerDMRI, UKF, and a number of other modules have empty package download files in the current nightly (tried SlicerRT, TCIABrowser, DSCMRIAnalysis, SlicerPathology, IntensitySegmenter). There are messages in the Slicer error log about empty archives, but regrettably the extension manager doesn't currently provide any prominent error message about this type of install failure.

 

I've opened a bug report: http://www.na-mic.org/Bug/view.php?id=4351

 

btw also found the newer releases crash when extension manager is called when older versions are in the .config folder.
moving them out solves the problem with latest releases after feb 1, tho inconvenient.

 

Not sure about this one -- I believe each version should have its own config sub-folder, and at least on Mac any downloaded extensions are stored inside the version-specific app bundle.

 

The Eigen notice is present on all platforms but should not affect functionality of either the main DMRI modules or UKF, when they are otherwise installed correctly. I'll see if I can find a way to fix the message.

 

Best,

Isaiah

 

 

On Tue, Feb 21, 2017 at 2:03 PM, Jeff Stevenson <[hidden email]> wrote:

Hi slicer folks, been following the latest module releases of DMRI and UKFT . i tried to download the same dev version today 4.7.0-2017-02-18 r25712.
on my mac the module install fine. on my 14.04 linux workstation i get a dependency error

“the extension Eigen could not be found. the extension
may not work properly.”

tractography display seems not to be installed when this happens, so DMRI/UKFT build on linux still has some issues.

cheers jeff

btw also found the newer releases crash when extension manager is called when older versions are in the .config folder.
moving them out solves the problem with latest releases after feb 1, tho inconvenient.

_______________________________________________
slicer-users mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-users
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/4.3/FAQ

 

 

 

 


_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/Nightly/Developers/FAQ