TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

Luis Ibanez
The Failing test of the Day is:

          Slicer3CLTest5

it fails due to memory leaks.

These leaks are now visible when
running the test locally as:

    ctest -R Slicer3CLTest5 -V


The command line of this test is
the following:

Slicer3-build/Slicer3 --test-mode --exec puts\ testing\ ,.\ exit\ 0


I'm certainly not Tcl expert...
but it seems that the command is constructed incorrectly.


I would have expected the --exec option to be followed
by the name of an executable or a shell command.


It would seem that this tests was a modified version of

          Slicer3CLTest4

whose command line is:

Slicer3-build/Slicer3 --test-mode --eval puts\ testing\ ,.\ exit\ 0

Which works correctly and passes
without any memory leaks.


Could a Tcl Guru please advice on the
correctness of the use of --exec ?


PS. We have 18 Failing tests.

By addressing one test every day
we will have a Green Dashboad by
the end of the AHM meeting in Janury   :-)


     Thanks


          Luis
_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
Reply | Threaded
Open this post in threaded view
|

Re: TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

Aucoin, Nicole
Hi Luis,

The exec and eval command line options for Slicer3 correspond to the tcl
commands of the same names.
exec expects arguments that make up a series of commands to pass to
subprocesses.
eval expects a series of arguments that it concatenates into tcl
commands to pass to the tcl interpreter.
In Slicer3.cxx, the eval arguments are passed to the tcl interpreter,
the slicer application is deleted and it returns.
Hmm, looking at the current code for exec, it looks like the actual call
to the tcl command is commented out, but it was set up to continue
starting up Slicer. That change is your check in, Luis, looking like a
side effect of a check in in the Testing dir.
If the exec command is actually run, I'm not surprised that it would be
generating leaks, as it will have any leaks that the full application
generates, while the eval command is returning immediately. I'll see if
I can add something to the test line to clean up properly, since it's
calling exit without any effort to clean up the application.

Nicole


Luis Ibanez wrote:

> The Failing test of the Day is:
>
>           Slicer3CLTest5
>
> it fails due to memory leaks.
>
> These leaks are now visible when
> running the test locally as:
>
>     ctest -R Slicer3CLTest5 -V
>
>
> The command line of this test is
> the following:
>
> Slicer3-build/Slicer3 --test-mode --exec puts\ testing\ ,.\ exit\ 0
>
>
> I'm certainly not Tcl expert...
> but it seems that the command is constructed incorrectly.
>
>
> I would have expected the --exec option to be followed
> by the name of an executable or a shell command.
>
>
> It would seem that this tests was a modified version of
>
>           Slicer3CLTest4
>
> whose command line is:
>
> Slicer3-build/Slicer3 --test-mode --eval puts\ testing\ ,.\ exit\ 0
>
> Which works correctly and passes
> without any memory leaks.
>
>
> Could a Tcl Guru please advice on the
> correctness of the use of --exec ?
>
>
> PS. We have 18 Failing tests.
>
> By addressing one test every day
> we will have a Green Dashboad by
> the end of the AHM meeting in Janury   :-)
>
>
>      Thanks
>
>
>           Luis
> _______________________________________________
> slicer-devel mailing list
> [hidden email]
> http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
> To unsubscribe: send email to [hidden email] with unsubscribe as the subject
>  

_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
Reply | Threaded
Open this post in threaded view
|

Re: TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

Aucoin, Nicole
Digging into this test, my conclusion is that it's not a very good test,
as calling exit in the exec command bypasses any memory release and
clean up in Slicer3.cxx and therefore will always lead to leak warnings.

Also, it looks like
vtkDebugLeaks::PrintCurrentLeaks();
only returns 1 or 0, not a count of the number of leaks, so the print
out from Slicer3 is a bit misleading when it's built in Release mode
(you don't see the leak list print out, just that there's 1 leak).

Nicole

Nicole Aucoin wrote:

> Hi Luis,
>
> The exec and eval command line options for Slicer3 correspond to the tcl
> commands of the same names.
> exec expects arguments that make up a series of commands to pass to
> subprocesses.
> eval expects a series of arguments that it concatenates into tcl
> commands to pass to the tcl interpreter.
> In Slicer3.cxx, the eval arguments are passed to the tcl interpreter,
> the slicer application is deleted and it returns.
> Hmm, looking at the current code for exec, it looks like the actual call
> to the tcl command is commented out, but it was set up to continue
> starting up Slicer. That change is your check in, Luis, looking like a
> side effect of a check in in the Testing dir.
> If the exec command is actually run, I'm not surprised that it would be
> generating leaks, as it will have any leaks that the full application
> generates, while the eval command is returning immediately. I'll see if
> I can add something to the test line to clean up properly, since it's
> calling exit without any effort to clean up the application.
>
> Nicole
>
>
> Luis Ibanez wrote:
>  
>> The Failing test of the Day is:
>>
>>           Slicer3CLTest5
>>
>> it fails due to memory leaks.
>>
>> These leaks are now visible when
>> running the test locally as:
>>
>>     ctest -R Slicer3CLTest5 -V
>>
>>
>> The command line of this test is
>> the following:
>>
>> Slicer3-build/Slicer3 --test-mode --exec puts\ testing\ ,.\ exit\ 0
>>
>>
>> I'm certainly not Tcl expert...
>> but it seems that the command is constructed incorrectly.
>>
>>
>> I would have expected the --exec option to be followed
>> by the name of an executable or a shell command.
>>
>>
>> It would seem that this tests was a modified version of
>>
>>           Slicer3CLTest4
>>
>> whose command line is:
>>
>> Slicer3-build/Slicer3 --test-mode --eval puts\ testing\ ,.\ exit\ 0
>>
>> Which works correctly and passes
>> without any memory leaks.
>>
>>
>> Could a Tcl Guru please advice on the
>> correctness of the use of --exec ?
>>
>>
>> PS. We have 18 Failing tests.
>>
>> By addressing one test every day
>> we will have a Green Dashboad by
>> the end of the AHM meeting in Janury   :-)
>>
>>
>>      Thanks
>>
>>
>>           Luis
>> _______________________________________________
>> slicer-devel mailing list
>> [hidden email]
>> http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
>> To unsubscribe: send email to [hidden email] with unsubscribe as the subject
>>  
>>    
>
> _______________________________________________
> slicer-devel mailing list
> [hidden email]
> http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
> To unsubscribe: send email to [hidden email] with unsubscribe as the subject
>  

_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
Reply | Threaded
Open this post in threaded view
|

Re: TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

Luis Ibanez
In reply to this post by Aucoin, Nicole
Hi Nicole,

Thanks a lot for enlightening what happens in this test.

My apologies for the commits in Slicer3.cxx

That was code intended for debugging only...

     (That is quite embarrassing....)      :-/

I just restored the lines:
http://viewvc.slicer.org/viewcvs.cgi/trunk/Applications/GUI/Slicer3.cxx?rev=11375&r1=11362&r2=11375

---

Could you please suggest what should be
added to the script in order to make a clean
return  ?

We could replace the current code in:

Applications/GUI/Testing/CMakeLists.txt:

add_test(Slicer3CLTest5 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
--exec "puts testing ,. exit 0")


Many thanks,


    Luis



-----------------------------------------------------------------------------
On Tue, Dec 22, 2009 at 10:57 AM, Nicole Aucoin <[hidden email]> wrote:

> Hi Luis,
>
> The exec and eval command line options for Slicer3 correspond to the tcl
> commands of the same names.
> exec expects arguments that make up a series of commands to pass to
> subprocesses.
> eval expects a series of arguments that it concatenates into tcl commands to
> pass to the tcl interpreter.
> In Slicer3.cxx, the eval arguments are passed to the tcl interpreter, the
> slicer application is deleted and it returns.
> Hmm, looking at the current code for exec, it looks like the actual call to
> the tcl command is commented out, but it was set up to continue starting up
> Slicer. That change is your check in, Luis, looking like a side effect of a
> check in in the Testing dir.
> If the exec command is actually run, I'm not surprised that it would be
> generating leaks, as it will have any leaks that the full application
> generates, while the eval command is returning immediately. I'll see if I
> can add something to the test line to clean up properly, since it's calling
> exit without any effort to clean up the application.
>
> Nicole
>
>
> Luis Ibanez wrote:
>>
>> The Failing test of the Day is:
>>
>>          Slicer3CLTest5
>>
>> it fails due to memory leaks.
>>
>> These leaks are now visible when
>> running the test locally as:
>>
>>    ctest -R Slicer3CLTest5 -V
>>
>>
>> The command line of this test is
>> the following:
>>
>> Slicer3-build/Slicer3 --test-mode --exec puts\ testing\ ,.\ exit\ 0
>>
>>
>> I'm certainly not Tcl expert...
>> but it seems that the command is constructed incorrectly.
>>
>>
>> I would have expected the --exec option to be followed
>> by the name of an executable or a shell command.
>>
>>
>> It would seem that this tests was a modified version of
>>
>>          Slicer3CLTest4
>>
>> whose command line is:
>>
>> Slicer3-build/Slicer3 --test-mode --eval puts\ testing\ ,.\ exit\ 0
>>
>> Which works correctly and passes
>> without any memory leaks.
>>
>>
>> Could a Tcl Guru please advice on the
>> correctness of the use of --exec ?
>>
>>
>> PS. We have 18 Failing tests.
>>
>> By addressing one test every day
>> we will have a Green Dashboad by
>> the end of the AHM meeting in Janury   :-)
>>
>>
>>     Thanks
>>
>>
>>          Luis
>> _______________________________________________
>> slicer-devel mailing list
>> [hidden email]
>> http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
>> To unsubscribe: send email to
>> [hidden email] with unsubscribe as the
>> subject
>>
>
>
_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
Reply | Threaded
Open this post in threaded view
|

Re: TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

Luis Ibanez
In reply to this post by Aucoin, Nicole
Hi Nicole,

Yes,
You are right, PrintCurrentLeaks() is only returning 0 or 1,
not the number of leaks as I was incorrectly assuming.

[ VTK/Common/vtkDebugLeaks.cxx, lines 315,330 and 367 ]


It would seems that vtkDebugLeaks could use a new
method such.

                GetCurrentNumberOfLeaks()



In the meantime I have fixed the message printed on exit,
to just say that there are leaks, instead of implying how
many memory leaks there are.
http://viewvc.slicer.org/viewcvs.cgi/trunk/Applications/GUI/Slicer3.cxx?rev=11375&r1=11375&r2=11376


    Thanks


         Luis


-------------------------------------------------------------------------------------------
On Tue, Dec 22, 2009 at 5:32 PM, Nicole Aucoin <[hidden email]> wrote:

> Digging into this test, my conclusion is that it's not a very good test, as
> calling exit in the exec command bypasses any memory release and clean up in
> Slicer3.cxx and therefore will always lead to leak warnings.
>
> Also, it looks like
> vtkDebugLeaks::PrintCurrentLeaks();
> only returns 1 or 0, not a count of the number of leaks, so the print out
> from Slicer3 is a bit misleading when it's built in Release mode (you don't
> see the leak list print out, just that there's 1 leak).
>
> Nicole
>
> Nicole Aucoin wrote:
>>
>> Hi Luis,
>>
>> The exec and eval command line options for Slicer3 correspond to the tcl
>> commands of the same names.
>> exec expects arguments that make up a series of commands to pass to
>> subprocesses.
>> eval expects a series of arguments that it concatenates into tcl commands
>> to pass to the tcl interpreter.
>> In Slicer3.cxx, the eval arguments are passed to the tcl interpreter, the
>> slicer application is deleted and it returns.
>> Hmm, looking at the current code for exec, it looks like the actual call
>> to the tcl command is commented out, but it was set up to continue starting
>> up Slicer. That change is your check in, Luis, looking like a side effect of
>> a check in in the Testing dir.
>> If the exec command is actually run, I'm not surprised that it would be
>> generating leaks, as it will have any leaks that the full application
>> generates, while the eval command is returning immediately. I'll see if I
>> can add something to the test line to clean up properly, since it's calling
>> exit without any effort to clean up the application.
>>
>> Nicole
>>
>>
>> Luis Ibanez wrote:
>>
>>>
>>> The Failing test of the Day is:
>>>
>>>          Slicer3CLTest5
>>>
>>> it fails due to memory leaks.
>>>
>>> These leaks are now visible when
>>> running the test locally as:
>>>
>>>    ctest -R Slicer3CLTest5 -V
>>>
>>>
>>> The command line of this test is
>>> the following:
>>>
>>> Slicer3-build/Slicer3 --test-mode --exec puts\ testing\ ,.\ exit\ 0
>>>
>>>
>>> I'm certainly not Tcl expert...
>>> but it seems that the command is constructed incorrectly.
>>>
>>>
>>> I would have expected the --exec option to be followed
>>> by the name of an executable or a shell command.
>>>
>>>
>>> It would seem that this tests was a modified version of
>>>
>>>          Slicer3CLTest4
>>>
>>> whose command line is:
>>>
>>> Slicer3-build/Slicer3 --test-mode --eval puts\ testing\ ,.\ exit\ 0
>>>
>>> Which works correctly and passes
>>> without any memory leaks.
>>>
>>>
>>> Could a Tcl Guru please advice on the
>>> correctness of the use of --exec ?
>>>
>>>
>>> PS. We have 18 Failing tests.
>>>
>>> By addressing one test every day
>>> we will have a Green Dashboad by
>>> the end of the AHM meeting in Janury   :-)
>>>
>>>
>>>     Thanks
>>>
>>>
>>>          Luis
>>> _______________________________________________
>>> slicer-devel mailing list
>>> [hidden email]
>>> http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
>>> To unsubscribe: send email to
>>> [hidden email] with unsubscribe as the
>>> subject
>>>
>>
>> _______________________________________________
>> slicer-devel mailing list
>> [hidden email]
>> http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
>> To unsubscribe: send email to
>> [hidden email] with unsubscribe as the
>> subject
>>
>
>
_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
Reply | Threaded
Open this post in threaded view
|

Re: TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

Sylvain Jaume-4
Hi Luis,

Did you mean the following?

if (numberOfLeaks != 0)
{
 cout << "There are memory leaks" << std::endl;
 res = EXIT_FAILURE;
}

I'm not sure as I'm only looking at the svn diff on the web interface.


Sylvain

On Dec 22, 2009, at 6:21 PM, Luis Ibanez <[hidden email]> wrote:

Hi Nicole,

Yes,
You are right, PrintCurrentLeaks() is only returning 0 or 1,
not the number of leaks as I was incorrectly assuming.

[ VTK/Common/vtkDebugLeaks.cxx, lines 315,330 and 367 ]


It would seems that vtkDebugLeaks could use a new
method such.

               GetCurrentNumberOfLeaks()



In the meantime I have fixed the message printed on exit,
to just say that there are leaks, instead of implying how
many memory leaks there are.
http://viewvc.slicer.org/viewcvs.cgi/trunk/Applications/GUI/Slicer3.cxx?rev=11375&r1=11375&r2=11376


   Thanks


        Luis


-------------------------------------------------------------------------------------------
On Tue, Dec 22, 2009 at 5:32 PM, Nicole Aucoin <[hidden email]> wrote:
Digging into this test, my conclusion is that it's not a very good test, as
calling exit in the exec command bypasses any memory release and clean up in
Slicer3.cxx and therefore will always lead to leak warnings.

Also, it looks like
vtkDebugLeaks::PrintCurrentLeaks();
only returns 1 or 0, not a count of the number of leaks, so the print out
from Slicer3 is a bit misleading when it's built in Release mode (you don't
see the leak list print out, just that there's 1 leak).

Nicole

Nicole Aucoin wrote:

Hi Luis,

The exec and eval command line options for Slicer3 correspond to the tcl
commands of the same names.
exec expects arguments that make up a series of commands to pass to
subprocesses.
eval expects a series of arguments that it concatenates into tcl commands
to pass to the tcl interpreter.
In Slicer3.cxx, the eval arguments are passed to the tcl interpreter, the
slicer application is deleted and it returns.
Hmm, looking at the current code for exec, it looks like the actual call
to the tcl command is commented out, but it was set up to continue starting
up Slicer. That change is your check in, Luis, looking like a side effect of
a check in in the Testing dir.
If the exec command is actually run, I'm not surprised that it would be
generating leaks, as it will have any leaks that the full application
generates, while the eval command is returning immediately. I'll see if I
can add something to the test line to clean up properly, since it's calling
exit without any effort to clean up the application.

Nicole


Luis Ibanez wrote:


The Failing test of the Day is:

         Slicer3CLTest5

it fails due to memory leaks.

These leaks are now visible when
running the test locally as:

   ctest -R Slicer3CLTest5 -V


The command line of this test is
the following:

Slicer3-build/Slicer3 --test-mode --exec puts\ testing\ ,.\ exit\ 0


I'm certainly not Tcl expert...
but it seems that the command is constructed incorrectly.


I would have expected the --exec option to be followed
by the name of an executable or a shell command.


It would seem that this tests was a modified version of

         Slicer3CLTest4

whose command line is:

Slicer3-build/Slicer3 --test-mode --eval puts\ testing\ ,.\ exit\ 0

Which works correctly and passes
without any memory leaks.


Could a Tcl Guru please advice on the
correctness of the use of --exec ?


PS. We have 18 Failing tests.

By addressing one test every day
we will have a Green Dashboad by
the end of the AHM meeting in Janury   :-)


    Thanks


         Luis
_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to
[hidden email] with unsubscribe as the
subject


_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to
[hidden email] with unsubscribe as the
subject



_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject

_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
Reply | Threaded
Open this post in threaded view
|

Re: TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

pieper
Administrator
In reply to this post by Luis Ibanez
Hi Luis -

Many thanks for stepping up as the champion of testing.  I would also
like to encourage everyone to take creating and fixing tests seriously.
  It will make all of our lives easier!

Regarding this simplest of all tests that you mentioned:

 > Could you please suggest what should be
 > added to the script in order to make a clean
 > return  ?
 >
 > We could replace the current code in:
 >
 > Applications/GUI/Testing/CMakeLists.txt:
 >
 > add_test(Slicer3CLTest5 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
 > --exec "puts testing ,. exit 0")
 >

This test simply lets slicer start up then exits cleanly.  This test is
a good way to see if the GUI startup up and shuts down without crashing
or leaking.  I don't think the test needs to be changed.  In fact, I'll
bet it touches a lot of code that has to all work right for this to pass
(module discovery, GUI startup, scene creation and deletion...).

The tests are all passing in the Slicer-3-4 branch (including this
simple one).  Things have gotten broken in the current trunk and need to
be fixed.

-----

Slicer developers may want to consider running the following handy
command lines before checking in code:

   ./Slicer3 --exec exit

(this checks that all of slicer will start and exit cleanly)

   ./Slicer3 --no-modules --exec exit

(this will test just the base of slicer: no modules will be loaded or
built).

   ./Slicer3 --ignore-module <ModuleName> --exec exit

(this will specifically leave out a module to help identify where a leak
  or crash is coming from.  Note that the module's GUI class constructor
will be called, but not the BuildGUI method, so can still generate test
failures even if you "--ignore" it).

With these you should be able to test

-----

Some notes on the testing script options for slicer (see the variants
from --help below - these are used in many of the tests currently
included in slicer):

* the standard tcl 'exit [code]' keyword is overridden.  Instead of
exiting right away, slicer's exit will return the flow of control to
after the StartApplication call in Slicer3.cxx.  This allows the regular
widget tear down sequence to happen.  The return code is passed back as
the exit code of the process.

* if a script is specified using one command line options, it will run
in the slicer interpreter (either the tcl or python interpreters,
depending on the form of the argument).  The *eval* form of the argument
avoids building the GUI, while the *exec* form builds the GUI and then
runs the script.

* the --test-mode argument disables any writing to the registry that
would otherwise occur (i.e. the screen size and layout are not saved on
exit)


Happy Holidays,
-Steve




    --evalpython <std::string>
      Like --exec, but doesn't load slicer first

    --execpython <std::string>
      Some Python code to execute after slicer loads. (note: cannot specify
      scene after --exec)

    -v <std::string>,  --eval <std::string>
      Like --exec, but doesn't load slicer first

    -e <std::string>,  --exec <std::string>
      Some code to execute after slicer loads. (note: cannot specify scene
      after --exec) (note: use ,. instead of ; between tcl statements)

    -p <std::string>,  --script <std::string>
      Script to execute after slicer loads

    -f <std::string>,  --file <std::string>
      Script to execute instead of loading slicer


_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
Reply | Threaded
Open this post in threaded view
|

Re: TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

Luis Ibanez
In reply to this post by Aucoin, Nicole
Hi Nicole, Steve

Are the following Slicer3-Testing instructions still valid:

http://www.slicer.org/slicerWiki/index.php/Slicer3:Testing


The basic recommended tests are:

<quote>

 ./Slicer3 --exec exit

(this checks that all of slicer will start and exit cleanly)

 ./Slicer3 --no-modules --exec exit

(this will test just the base of slicer: no modules will be loaded or built).

 ./Slicer3 --ignore-module <ModuleName> --exec exit

</quote>


Thanks for any advice,


     Luis


----------------------------------------------------------------------------------------
On Tue, Dec 22, 2009 at 5:32 PM, Nicole Aucoin <[hidden email]> wrote:

> Digging into this test, my conclusion is that it's not a very good test, as
> calling exit in the exec command bypasses any memory release and clean up in
> Slicer3.cxx and therefore will always lead to leak warnings.
>
> Also, it looks like
> vtkDebugLeaks::PrintCurrentLeaks();
> only returns 1 or 0, not a count of the number of leaks, so the print out
> from Slicer3 is a bit misleading when it's built in Release mode (you don't
> see the leak list print out, just that there's 1 leak).
>
> Nicole
>
> Nicole Aucoin wrote:
>>
>> Hi Luis,
>>
>> The exec and eval command line options for Slicer3 correspond to the tcl
>> commands of the same names.
>> exec expects arguments that make up a series of commands to pass to
>> subprocesses.
>> eval expects a series of arguments that it concatenates into tcl commands
>> to pass to the tcl interpreter.
>> In Slicer3.cxx, the eval arguments are passed to the tcl interpreter, the
>> slicer application is deleted and it returns.
>> Hmm, looking at the current code for exec, it looks like the actual call
>> to the tcl command is commented out, but it was set up to continue starting
>> up Slicer. That change is your check in, Luis, looking like a side effect of
>> a check in in the Testing dir.
>> If the exec command is actually run, I'm not surprised that it would be
>> generating leaks, as it will have any leaks that the full application
>> generates, while the eval command is returning immediately. I'll see if I
>> can add something to the test line to clean up properly, since it's calling
>> exit without any effort to clean up the application.
>>
>> Nicole
>>
>>
>> Luis Ibanez wrote:
>>
>>>
>>> The Failing test of the Day is:
>>>
>>>          Slicer3CLTest5
>>>
>>> it fails due to memory leaks.
>>>
>>> These leaks are now visible when
>>> running the test locally as:
>>>
>>>    ctest -R Slicer3CLTest5 -V
>>>
>>>
>>> The command line of this test is
>>> the following:
>>>
>>> Slicer3-build/Slicer3 --test-mode --exec puts\ testing\ ,.\ exit\ 0
>>>
>>>
>>> I'm certainly not Tcl expert...
>>> but it seems that the command is constructed incorrectly.
>>>
>>>
>>> I would have expected the --exec option to be followed
>>> by the name of an executable or a shell command.
>>>
>>>
>>> It would seem that this tests was a modified version of
>>>
>>>          Slicer3CLTest4
>>>
>>> whose command line is:
>>>
>>> Slicer3-build/Slicer3 --test-mode --eval puts\ testing\ ,.\ exit\ 0
>>>
>>> Which works correctly and passes
>>> without any memory leaks.
>>>
>>>
>>> Could a Tcl Guru please advice on the
>>> correctness of the use of --exec ?
>>>
>>>
>>> PS. We have 18 Failing tests.
>>>
>>> By addressing one test every day
>>> we will have a Green Dashboad by
>>> the end of the AHM meeting in Janury   :-)
>>>
>>>
>>>     Thanks
>>>
>>>
>>>          Luis
>>> _______________________________________________
>>> slicer-devel mailing list
>>> [hidden email]
>>> http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
>>> To unsubscribe: send email to
>>> [hidden email] with unsubscribe as the
>>> subject
>>>
>>
>> _______________________________________________
>> slicer-devel mailing list
>> [hidden email]
>> http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
>> To unsubscribe: send email to
>> [hidden email] with unsubscribe as the
>> subject
>>
>
>
_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
Reply | Threaded
Open this post in threaded view
|

Re: TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

Luis Ibanez
In reply to this post by pieper
Hi Steve,

Thanks a lot for providing more guidelines for testing,

I was actually wondering a moment ago if the instructions at
http://www.slicer.org/slicerWiki/index.php/Slicer3:Testing
were still valid.

It seems that some of these suggested tests
are the ones executed  by the tests:

    ctest -R Slicer3CLTest -N

 86/ 96 Testing Slicer3CLTest1
 87/ 96 Testing Slicer3CLTest2
 88/ 96 Testing Slicer3CLTest3
 89/ 96 Testing Slicer3CLTest4
 90/ 96 Testing Slicer3CLTest5
 91/ 96 Testing Slicer3CLTest6
 92/ 96 Testing Slicer3CLTest7

But.. it also seems that we have not all of the
ones suggested in
http://www.slicer.org/slicerWiki/index.php/Slicer3:Testing
and not all the ones in your email.

Shouldn't we add the missing ones ?


The current ones are:

    ctest -R Slicer3CLTest -N -V


86/ 96 Testing Slicer3CLTest1

Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode --help
 87/ 96 Testing Slicer3CLTest2

Test command: /home/ibanez/src/Slicer3-build/Slicer3 --xml --test-mode
 88/ 96 Testing Slicer3CLTest3

Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode -f
/home/ibanez/src/Slicer3/Applications/GUI/Testing/TestScript.tcl
 89/ 96 Testing Slicer3CLTest4

Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
--eval puts\ testing\ ,.\ exit\ 0
 90/ 96 Testing Slicer3CLTest5

Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
--exec puts\ testing\ ,.\ exit\ 0
 91/ 96 Testing Slicer3CLTest6

Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
--script /home/ibanez/src/Slicer3/Applications/GUI/Testing/TestScript.tcl
 92/ 96 Testing Slicer3CLTest7

Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
--script /home/ibanez/src/Slicer3/Applications/GUI/Testing/LoadSceneTest.tcl
 93/ 96 Testing Slicer3ScrollTest

Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
--script /home/ibanez/src/Slicer3/Applications/GUI/Testing/ScrollTesting.tcl
 94/ 96 Testing Slicer3MRMLUndo

Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
--no-splash -f /home/ibanez/src/Slicer3-build/share/MRML/Testing/testUndo.tcl
 95/ 96 Testing Slicer3MRMLVolume

Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
--no-splash -f /home/ibanez/src/Slicer3-build/share/MRML/Testing/testVolume.tcl


---

Also,

Given the large amount of memory leaks currently
present in Slicer3, I propose to engage in a campaign
of vtkSmartPointer<>  adoption.

Are there any reasons for not using vtkSmartPointers
massively in Slicer ?


     Thanks,


          Luis


------------------------------------------------------------------------------------
On Tue, Dec 22, 2009 at 7:42 PM, Steve Pieper <[hidden email]> wrote:

> Hi Luis -
>
> Many thanks for stepping up as the champion of testing.  I would also like
> to encourage everyone to take creating and fixing tests seriously.  It will
> make all of our lives easier!
>
> Regarding this simplest of all tests that you mentioned:
>
>> Could you please suggest what should be
>> added to the script in order to make a clean
>> return  ?
>>
>> We could replace the current code in:
>>
>> Applications/GUI/Testing/CMakeLists.txt:
>>
>> add_test(Slicer3CLTest5 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
>> --exec "puts testing ,. exit 0")
>>
>
> This test simply lets slicer start up then exits cleanly.  This test is a
> good way to see if the GUI startup up and shuts down without crashing or
> leaking.  I don't think the test needs to be changed.  In fact, I'll bet it
> touches a lot of code that has to all work right for this to pass (module
> discovery, GUI startup, scene creation and deletion...).
>
> The tests are all passing in the Slicer-3-4 branch (including this simple
> one).  Things have gotten broken in the current trunk and need to be fixed.
>
> -----
>
> Slicer developers may want to consider running the following handy command
> lines before checking in code:
>
>  ./Slicer3 --exec exit
>
> (this checks that all of slicer will start and exit cleanly)
>
>  ./Slicer3 --no-modules --exec exit
>
> (this will test just the base of slicer: no modules will be loaded or
> built).
>
>  ./Slicer3 --ignore-module <ModuleName> --exec exit
>
> (this will specifically leave out a module to help identify where a leak  or
> crash is coming from.  Note that the module's GUI class constructor will be
> called, but not the BuildGUI method, so can still generate test failures
> even if you "--ignore" it).
>
> With these you should be able to test
>
> -----
>
> Some notes on the testing script options for slicer (see the variants from
> --help below - these are used in many of the tests currently included in
> slicer):
>
> * the standard tcl 'exit [code]' keyword is overridden.  Instead of exiting
> right away, slicer's exit will return the flow of control to after the
> StartApplication call in Slicer3.cxx.  This allows the regular widget tear
> down sequence to happen.  The return code is passed back as the exit code of
> the process.
>
> * if a script is specified using one command line options, it will run in
> the slicer interpreter (either the tcl or python interpreters, depending on
> the form of the argument).  The *eval* form of the argument avoids building
> the GUI, while the *exec* form builds the GUI and then runs the script.
>
> * the --test-mode argument disables any writing to the registry that would
> otherwise occur (i.e. the screen size and layout are not saved on exit)
>
>
> Happy Holidays,
> -Steve
>
>
>
>
>   --evalpython <std::string>
>     Like --exec, but doesn't load slicer first
>
>   --execpython <std::string>
>     Some Python code to execute after slicer loads. (note: cannot specify
>     scene after --exec)
>
>   -v <std::string>,  --eval <std::string>
>     Like --exec, but doesn't load slicer first
>
>   -e <std::string>,  --exec <std::string>
>     Some code to execute after slicer loads. (note: cannot specify scene
>     after --exec) (note: use ,. instead of ; between tcl statements)
>
>   -p <std::string>,  --script <std::string>
>     Script to execute after slicer loads
>
>   -f <std::string>,  --file <std::string>
>     Script to execute instead of loading slicer
>
>
>
_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
Reply | Threaded
Open this post in threaded view
|

Re: TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

Aucoin, Nicole
In reply to this post by pieper
I think it was the over ridden exit() command that I wasn't seeing, as
from the code it looks like Slicer isn't  getting a chance to clean up
before exiting if you pass exit to the exec command.  With --no-modules
--exec exit, there's an event broker leak that I wasn't able to track
down, but I'll try to clean up the modules that are leaking.

Nicole

Steve Pieper wrote:

> Hi Luis -
>
> Many thanks for stepping up as the champion of testing.  I would also
> like to encourage everyone to take creating and fixing tests
> seriously.  It will make all of our lives easier!
>
> Regarding this simplest of all tests that you mentioned:
>
> > Could you please suggest what should be
> > added to the script in order to make a clean
> > return  ?
> >
> > We could replace the current code in:
> >
> > Applications/GUI/Testing/CMakeLists.txt:
> >
> > add_test(Slicer3CLTest5 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
> > --exec "puts testing ,. exit 0")
> >
>
> This test simply lets slicer start up then exits cleanly.  This test
> is a good way to see if the GUI startup up and shuts down without
> crashing or leaking.  I don't think the test needs to be changed.  In
> fact, I'll bet it touches a lot of code that has to all work right for
> this to pass (module discovery, GUI startup, scene creation and
> deletion...).
>
> The tests are all passing in the Slicer-3-4 branch (including this
> simple one).  Things have gotten broken in the current trunk and need
> to be fixed.
>
> -----
>
> Slicer developers may want to consider running the following handy
> command lines before checking in code:
>
>   ./Slicer3 --exec exit
>
> (this checks that all of slicer will start and exit cleanly)
>
>   ./Slicer3 --no-modules --exec exit
>
> (this will test just the base of slicer: no modules will be loaded or
> built).
>
>   ./Slicer3 --ignore-module <ModuleName> --exec exit
>
> (this will specifically leave out a module to help identify where a
> leak  or crash is coming from.  Note that the module's GUI class
> constructor will be called, but not the BuildGUI method, so can still
> generate test failures even if you "--ignore" it).
>
> With these you should be able to test
>
> -----
>
> Some notes on the testing script options for slicer (see the variants
> from --help below - these are used in many of the tests currently
> included in slicer):
>
> * the standard tcl 'exit [code]' keyword is overridden.  Instead of
> exiting right away, slicer's exit will return the flow of control to
> after the StartApplication call in Slicer3.cxx.  This allows the
> regular widget tear down sequence to happen.  The return code is
> passed back as the exit code of the process.
>
> * if a script is specified using one command line options, it will run
> in the slicer interpreter (either the tcl or python interpreters,
> depending on the form of the argument).  The *eval* form of the
> argument avoids building the GUI, while the *exec* form builds the GUI
> and then runs the script.
>
> * the --test-mode argument disables any writing to the registry that
> would otherwise occur (i.e. the screen size and layout are not saved
> on exit)
>
>
> Happy Holidays,
> -Steve
>
>
>
>
>    --evalpython <std::string>
>      Like --exec, but doesn't load slicer first
>
>    --execpython <std::string>
>      Some Python code to execute after slicer loads. (note: cannot
> specify
>      scene after --exec)
>
>    -v <std::string>,  --eval <std::string>
>      Like --exec, but doesn't load slicer first
>
>    -e <std::string>,  --exec <std::string>
>      Some code to execute after slicer loads. (note: cannot specify scene
>      after --exec) (note: use ,. instead of ; between tcl statements)
>
>    -p <std::string>,  --script <std::string>
>      Script to execute after slicer loads
>
>    -f <std::string>,  --file <std::string>
>      Script to execute instead of loading slicer
>
>

_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
Reply | Threaded
Open this post in threaded view
|

Re: TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

pieper
Administrator
In reply to this post by Luis Ibanez
Hi Luis -

Yes, it would make a lot of sense to add the following as a test:

  ./Slicer3 --no-modules --exec exit

On the current trunk, this test fails :(  No doubt as a result of some
recent checkins.  Having this checked automatically will help (assuming
people pay attention when the test stops working!!!).

Also, it would be great to add a set of tests, one for each module that
is bundled with the slicer build of the form:

  ./Slicer3 --ignore-module <ModuleName> --exec exit

The other command line arguments would also be great to test.


Regarding vtkSmartPointer in slicer, Jim has written some excellent
guidelines for how it should be used.

http://www.slicer.org/slicerWiki/index.php/Slicer3:Memory_Management

The primary development of slicer has been to copy the style of vtk for
consistency and the New()/Delete() pattern is widely used inside vtk.
Making a change to smart pointers would be a lot of work and probably
not the most efficient way to fix the current memory leaks.  But if
someone wants to take this on (and does it well) I wouldn't object.

Best,
Steve



Luis Ibanez wrote:

> Hi Steve,
>
> Thanks a lot for providing more guidelines for testing,
>
> I was actually wondering a moment ago if the instructions at
> http://www.slicer.org/slicerWiki/index.php/Slicer3:Testing
> were still valid.
>
> It seems that some of these suggested tests
> are the ones executed  by the tests:
>
>     ctest -R Slicer3CLTest -N
>
>  86/ 96 Testing Slicer3CLTest1
>  87/ 96 Testing Slicer3CLTest2
>  88/ 96 Testing Slicer3CLTest3
>  89/ 96 Testing Slicer3CLTest4
>  90/ 96 Testing Slicer3CLTest5
>  91/ 96 Testing Slicer3CLTest6
>  92/ 96 Testing Slicer3CLTest7
>
> But.. it also seems that we have not all of the
> ones suggested in
> http://www.slicer.org/slicerWiki/index.php/Slicer3:Testing
> and not all the ones in your email.
>
> Shouldn't we add the missing ones ?
>
>
> The current ones are:
>
>     ctest -R Slicer3CLTest -N -V
>
>
> 86/ 96 Testing Slicer3CLTest1
>
> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode --help
>  87/ 96 Testing Slicer3CLTest2
>
> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --xml --test-mode
>  88/ 96 Testing Slicer3CLTest3
>
> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode -f
> /home/ibanez/src/Slicer3/Applications/GUI/Testing/TestScript.tcl
>  89/ 96 Testing Slicer3CLTest4
>
> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
> --eval puts\ testing\ ,.\ exit\ 0
>  90/ 96 Testing Slicer3CLTest5
>
> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
> --exec puts\ testing\ ,.\ exit\ 0
>  91/ 96 Testing Slicer3CLTest6
>
> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
> --script /home/ibanez/src/Slicer3/Applications/GUI/Testing/TestScript.tcl
>  92/ 96 Testing Slicer3CLTest7
>
> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
> --script /home/ibanez/src/Slicer3/Applications/GUI/Testing/LoadSceneTest.tcl
>  93/ 96 Testing Slicer3ScrollTest
>
> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
> --script /home/ibanez/src/Slicer3/Applications/GUI/Testing/ScrollTesting.tcl
>  94/ 96 Testing Slicer3MRMLUndo
>
> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
> --no-splash -f /home/ibanez/src/Slicer3-build/share/MRML/Testing/testUndo.tcl
>  95/ 96 Testing Slicer3MRMLVolume
>
> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
> --no-splash -f /home/ibanez/src/Slicer3-build/share/MRML/Testing/testVolume.tcl
>
>
> ---
>
> Also,
>
> Given the large amount of memory leaks currently
> present in Slicer3, I propose to engage in a campaign
> of vtkSmartPointer<>  adoption.
>
> Are there any reasons for not using vtkSmartPointers
> massively in Slicer ?
>
>
>      Thanks,
>
>
>           Luis
>
>
> ------------------------------------------------------------------------------------
> On Tue, Dec 22, 2009 at 7:42 PM, Steve Pieper <[hidden email]> wrote:
>> Hi Luis -
>>
>> Many thanks for stepping up as the champion of testing.  I would also like
>> to encourage everyone to take creating and fixing tests seriously.  It will
>> make all of our lives easier!
>>
>> Regarding this simplest of all tests that you mentioned:
>>
>>> Could you please suggest what should be
>>> added to the script in order to make a clean
>>> return  ?
>>>
>>> We could replace the current code in:
>>>
>>> Applications/GUI/Testing/CMakeLists.txt:
>>>
>>> add_test(Slicer3CLTest5 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
>>> --exec "puts testing ,. exit 0")
>>>
>> This test simply lets slicer start up then exits cleanly.  This test is a
>> good way to see if the GUI startup up and shuts down without crashing or
>> leaking.  I don't think the test needs to be changed.  In fact, I'll bet it
>> touches a lot of code that has to all work right for this to pass (module
>> discovery, GUI startup, scene creation and deletion...).
>>
>> The tests are all passing in the Slicer-3-4 branch (including this simple
>> one).  Things have gotten broken in the current trunk and need to be fixed.
>>
>> -----
>>
>> Slicer developers may want to consider running the following handy command
>> lines before checking in code:
>>
>>  ./Slicer3 --exec exit
>>
>> (this checks that all of slicer will start and exit cleanly)
>>
>>  ./Slicer3 --no-modules --exec exit
>>
>> (this will test just the base of slicer: no modules will be loaded or
>> built).
>>
>>  ./Slicer3 --ignore-module <ModuleName> --exec exit
>>
>> (this will specifically leave out a module to help identify where a leak  or
>> crash is coming from.  Note that the module's GUI class constructor will be
>> called, but not the BuildGUI method, so can still generate test failures
>> even if you "--ignore" it).
>>
>> With these you should be able to test
>>
>> -----
>>
>> Some notes on the testing script options for slicer (see the variants from
>> --help below - these are used in many of the tests currently included in
>> slicer):
>>
>> * the standard tcl 'exit [code]' keyword is overridden.  Instead of exiting
>> right away, slicer's exit will return the flow of control to after the
>> StartApplication call in Slicer3.cxx.  This allows the regular widget tear
>> down sequence to happen.  The return code is passed back as the exit code of
>> the process.
>>
>> * if a script is specified using one command line options, it will run in
>> the slicer interpreter (either the tcl or python interpreters, depending on
>> the form of the argument).  The *eval* form of the argument avoids building
>> the GUI, while the *exec* form builds the GUI and then runs the script.
>>
>> * the --test-mode argument disables any writing to the registry that would
>> otherwise occur (i.e. the screen size and layout are not saved on exit)
>>
>>
>> Happy Holidays,
>> -Steve
>>
>>
>>
>>
>>   --evalpython <std::string>
>>     Like --exec, but doesn't load slicer first
>>
>>   --execpython <std::string>
>>     Some Python code to execute after slicer loads. (note: cannot specify
>>     scene after --exec)
>>
>>   -v <std::string>,  --eval <std::string>
>>     Like --exec, but doesn't load slicer first
>>
>>   -e <std::string>,  --exec <std::string>
>>     Some code to execute after slicer loads. (note: cannot specify scene
>>     after --exec) (note: use ,. instead of ; between tcl statements)
>>
>>   -p <std::string>,  --script <std::string>
>>     Script to execute after slicer loads
>>
>>   -f <std::string>,  --file <std::string>
>>     Script to execute instead of loading slicer
>>
>>
>>
_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
Reply | Threaded
Open this post in threaded view
|

Re: TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

Luis Ibanez
On Wed, Dec 23, 2009 at 2:06 PM, Steve Pieper <[hidden email]> wrote:
> Hi Luis -
>
> Yes, it would make a lot of sense to add the following as a test:
>
>  ./Slicer3 --no-modules --exec exit
>

The test has been added
http://viewvc.slicer.org/viewcvs.cgi/trunk/Applications/GUI/Testing/CMakeLists.txt?rev=11378&sortby=date&r2=11378&r1=11255

They are:

* Slicer3CLTest8
* Slicer3CLTest9

add_test(Slicer3CLTest8 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode --no-modules)
add_test(Slicer3CLTest9 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
--no-modules --exec "exit")

As you pointed out, both are failing

BTW,
I assume that we also need the "--test-mode" flag,
is that correct ?


> On the current trunk, this test fails :(  No doubt as a result of some
> recent checkins.  Having this checked automatically will help (assuming
> people pay attention when the test stops working!!!).
>

Could you provide any guess as to how recent the
offending commit may have been ?

Cleaning up the Dashboard is
a critical task at this point...


> Also, it would be great to add a set of tests, one for each module that is
> bundled with the slicer build of the form:
>
>  ./Slicer3 --ignore-module <ModuleName> --exec exit
>
> The other command line arguments would also be great to test.
>
>
> Regarding vtkSmartPointer in slicer, Jim has written some excellent
> guidelines for how it should be used.
>
> http://www.slicer.org/slicerWiki/index.php/Slicer3:Memory_Management
>
> The primary development of slicer has been to copy the style of vtk for
> consistency and the New()/Delete() pattern is widely used inside vtk. Making
> a change to smart pointers would be a lot of work and probably not the most
> efficient way to fix the current memory leaks.  But if someone wants to take
> this on (and does it well) I wouldn't object.
>
> Best,
> Steve
>
>
>
> Luis Ibanez wrote:
>>
>> Hi Steve,
>>
>> Thanks a lot for providing more guidelines for testing,
>>
>> I was actually wondering a moment ago if the instructions at
>> http://www.slicer.org/slicerWiki/index.php/Slicer3:Testing
>> were still valid.
>>
>> It seems that some of these suggested tests
>> are the ones executed  by the tests:
>>
>>    ctest -R Slicer3CLTest -N
>>
>>  86/ 96 Testing Slicer3CLTest1
>>  87/ 96 Testing Slicer3CLTest2
>>  88/ 96 Testing Slicer3CLTest3
>>  89/ 96 Testing Slicer3CLTest4
>>  90/ 96 Testing Slicer3CLTest5
>>  91/ 96 Testing Slicer3CLTest6
>>  92/ 96 Testing Slicer3CLTest7
>>
>> But.. it also seems that we have not all of the
>> ones suggested in
>> http://www.slicer.org/slicerWiki/index.php/Slicer3:Testing
>> and not all the ones in your email.
>>
>> Shouldn't we add the missing ones ?
>>
>>
>> The current ones are:
>>
>>    ctest -R Slicer3CLTest -N -V
>>
>>
>> 86/ 96 Testing Slicer3CLTest1
>>
>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode --help
>>  87/ 96 Testing Slicer3CLTest2
>>
>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --xml --test-mode
>>  88/ 96 Testing Slicer3CLTest3
>>
>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode -f
>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/TestScript.tcl
>>  89/ 96 Testing Slicer3CLTest4
>>
>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>> --eval puts\ testing\ ,.\ exit\ 0
>>  90/ 96 Testing Slicer3CLTest5
>>
>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>> --exec puts\ testing\ ,.\ exit\ 0
>>  91/ 96 Testing Slicer3CLTest6
>>
>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>> --script /home/ibanez/src/Slicer3/Applications/GUI/Testing/TestScript.tcl
>>  92/ 96 Testing Slicer3CLTest7
>>
>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>> --script
>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/LoadSceneTest.tcl
>>  93/ 96 Testing Slicer3ScrollTest
>>
>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>> --script
>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/ScrollTesting.tcl
>>  94/ 96 Testing Slicer3MRMLUndo
>>
>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>> --no-splash -f
>> /home/ibanez/src/Slicer3-build/share/MRML/Testing/testUndo.tcl
>>  95/ 96 Testing Slicer3MRMLVolume
>>
>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>> --no-splash -f
>> /home/ibanez/src/Slicer3-build/share/MRML/Testing/testVolume.tcl
>>
>>
>> ---
>>
>> Also,
>>
>> Given the large amount of memory leaks currently
>> present in Slicer3, I propose to engage in a campaign
>> of vtkSmartPointer<>  adoption.
>>
>> Are there any reasons for not using vtkSmartPointers
>> massively in Slicer ?
>>
>>
>>     Thanks,
>>
>>
>>          Luis
>>
>>
>>
>> ------------------------------------------------------------------------------------
>> On Tue, Dec 22, 2009 at 7:42 PM, Steve Pieper <[hidden email]>
>> wrote:
>>>
>>> Hi Luis -
>>>
>>> Many thanks for stepping up as the champion of testing.  I would also
>>> like
>>> to encourage everyone to take creating and fixing tests seriously.  It
>>> will
>>> make all of our lives easier!
>>>
>>> Regarding this simplest of all tests that you mentioned:
>>>
>>>> Could you please suggest what should be
>>>> added to the script in order to make a clean
>>>> return  ?
>>>>
>>>> We could replace the current code in:
>>>>
>>>> Applications/GUI/Testing/CMakeLists.txt:
>>>>
>>>> add_test(Slicer3CLTest5 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
>>>> --exec "puts testing ,. exit 0")
>>>>
>>> This test simply lets slicer start up then exits cleanly.  This test is a
>>> good way to see if the GUI startup up and shuts down without crashing or
>>> leaking.  I don't think the test needs to be changed.  In fact, I'll bet
>>> it
>>> touches a lot of code that has to all work right for this to pass (module
>>> discovery, GUI startup, scene creation and deletion...).
>>>
>>> The tests are all passing in the Slicer-3-4 branch (including this simple
>>> one).  Things have gotten broken in the current trunk and need to be
>>> fixed.
>>>
>>> -----
>>>
>>> Slicer developers may want to consider running the following handy
>>> command
>>> lines before checking in code:
>>>
>>>  ./Slicer3 --exec exit
>>>
>>> (this checks that all of slicer will start and exit cleanly)
>>>
>>>  ./Slicer3 --no-modules --exec exit
>>>
>>> (this will test just the base of slicer: no modules will be loaded or
>>> built).
>>>
>>>  ./Slicer3 --ignore-module <ModuleName> --exec exit
>>>
>>> (this will specifically leave out a module to help identify where a leak
>>>  or
>>> crash is coming from.  Note that the module's GUI class constructor will
>>> be
>>> called, but not the BuildGUI method, so can still generate test failures
>>> even if you "--ignore" it).
>>>
>>> With these you should be able to test
>>>
>>> -----
>>>
>>> Some notes on the testing script options for slicer (see the variants
>>> from
>>> --help below - these are used in many of the tests currently included in
>>> slicer):
>>>
>>> * the standard tcl 'exit [code]' keyword is overridden.  Instead of
>>> exiting
>>> right away, slicer's exit will return the flow of control to after the
>>> StartApplication call in Slicer3.cxx.  This allows the regular widget
>>> tear
>>> down sequence to happen.  The return code is passed back as the exit code
>>> of
>>> the process.
>>>
>>> * if a script is specified using one command line options, it will run in
>>> the slicer interpreter (either the tcl or python interpreters, depending
>>> on
>>> the form of the argument).  The *eval* form of the argument avoids
>>> building
>>> the GUI, while the *exec* form builds the GUI and then runs the script.
>>>
>>> * the --test-mode argument disables any writing to the registry that
>>> would
>>> otherwise occur (i.e. the screen size and layout are not saved on exit)
>>>
>>>
>>> Happy Holidays,
>>> -Steve
>>>
>>>
>>>
>>>
>>>  --evalpython <std::string>
>>>    Like --exec, but doesn't load slicer first
>>>
>>>  --execpython <std::string>
>>>    Some Python code to execute after slicer loads. (note: cannot specify
>>>    scene after --exec)
>>>
>>>  -v <std::string>,  --eval <std::string>
>>>    Like --exec, but doesn't load slicer first
>>>
>>>  -e <std::string>,  --exec <std::string>
>>>    Some code to execute after slicer loads. (note: cannot specify scene
>>>    after --exec) (note: use ,. instead of ; between tcl statements)
>>>
>>>  -p <std::string>,  --script <std::string>
>>>    Script to execute after slicer loads
>>>
>>>  -f <std::string>,  --file <std::string>
>>>    Script to execute instead of loading slicer
>>>
>>>
>>>
>
_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
Reply | Threaded
Open this post in threaded view
|

Re: TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

pieper
Administrator
Hi Luis -

Thanks for the new tests - now let's get them passing :)

Yes, we alway use the --test-mode flag on the cmake-based tests.  It
skips the splash screen and also doesn't save any changes to the
registry (and perhaps a few other things).

Here's what I get on my build:

> % ./Slicer3 --no-modules --exec exit
> Setting up launch environment...
> Starting Slicer: /Users/pieper/slicer3/latest/Slicer3-build
> vtkDebugLeaks has detected LEAKS!
> Class "vtkOutputWindow" has 1 instance still around.
> Class "vtkEventBroker" has 1 instance still around.
> Class "vtkFreeTypeUtilities" has 1 instance still around.
> Class "vtkStringArray" has 2 instances still around.
> Class "vtkTimerLog" has 1 instance still around.
>
> There are memory leaks
>
> child process exited abnormally
>

The vtkFreeTypeUtilities class is odd - perhaps it is created by the
vtkOutputWindow.  The vtkStringArray and vtkTimerLog might also be
associated with this.

JC recently introduced a new singleton pattern for the vtkEventBroker so
I'm not sure why that is leaking.

Typically even a single missing Delete() can lead to a whole cascade of
leaks.  Tracking them down is a bit of a puzzle.  Satisfying when they
get cleared up, but frustrating that they keep re-appearing.  Perhaps
the vtkSmartPointer conversion isn't such a bad idea...

-Steve

Luis Ibanez wrote:

> On Wed, Dec 23, 2009 at 2:06 PM, Steve Pieper <[hidden email]> wrote:
>> Hi Luis -
>>
>> Yes, it would make a lot of sense to add the following as a test:
>>
>>  ./Slicer3 --no-modules --exec exit
>>
>
> The test has been added
> http://viewvc.slicer.org/viewcvs.cgi/trunk/Applications/GUI/Testing/CMakeLists.txt?rev=11378&sortby=date&r2=11378&r1=11255
>
> They are:
>
> * Slicer3CLTest8
> * Slicer3CLTest9
>
> add_test(Slicer3CLTest8 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode --no-modules)
> add_test(Slicer3CLTest9 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
> --no-modules --exec "exit")
>
> As you pointed out, both are failing
>
> BTW,
> I assume that we also need the "--test-mode" flag,
> is that correct ?
>
>
>> On the current trunk, this test fails :(  No doubt as a result of some
>> recent checkins.  Having this checked automatically will help (assuming
>> people pay attention when the test stops working!!!).
>>
>
> Could you provide any guess as to how recent the
> offending commit may have been ?
>
> Cleaning up the Dashboard is
> a critical task at this point...
>
>
>> Also, it would be great to add a set of tests, one for each module that is
>> bundled with the slicer build of the form:
>>
>>  ./Slicer3 --ignore-module <ModuleName> --exec exit
>>
>> The other command line arguments would also be great to test.
>>
>>
>> Regarding vtkSmartPointer in slicer, Jim has written some excellent
>> guidelines for how it should be used.
>>
>> http://www.slicer.org/slicerWiki/index.php/Slicer3:Memory_Management
>>
>> The primary development of slicer has been to copy the style of vtk for
>> consistency and the New()/Delete() pattern is widely used inside vtk. Making
>> a change to smart pointers would be a lot of work and probably not the most
>> efficient way to fix the current memory leaks.  But if someone wants to take
>> this on (and does it well) I wouldn't object.
>>
>> Best,
>> Steve
>>
>>
>>
>> Luis Ibanez wrote:
>>> Hi Steve,
>>>
>>> Thanks a lot for providing more guidelines for testing,
>>>
>>> I was actually wondering a moment ago if the instructions at
>>> http://www.slicer.org/slicerWiki/index.php/Slicer3:Testing
>>> were still valid.
>>>
>>> It seems that some of these suggested tests
>>> are the ones executed  by the tests:
>>>
>>>    ctest -R Slicer3CLTest -N
>>>
>>>  86/ 96 Testing Slicer3CLTest1
>>>  87/ 96 Testing Slicer3CLTest2
>>>  88/ 96 Testing Slicer3CLTest3
>>>  89/ 96 Testing Slicer3CLTest4
>>>  90/ 96 Testing Slicer3CLTest5
>>>  91/ 96 Testing Slicer3CLTest6
>>>  92/ 96 Testing Slicer3CLTest7
>>>
>>> But.. it also seems that we have not all of the
>>> ones suggested in
>>> http://www.slicer.org/slicerWiki/index.php/Slicer3:Testing
>>> and not all the ones in your email.
>>>
>>> Shouldn't we add the missing ones ?
>>>
>>>
>>> The current ones are:
>>>
>>>    ctest -R Slicer3CLTest -N -V
>>>
>>>
>>> 86/ 96 Testing Slicer3CLTest1
>>>
>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode --help
>>>  87/ 96 Testing Slicer3CLTest2
>>>
>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --xml --test-mode
>>>  88/ 96 Testing Slicer3CLTest3
>>>
>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode -f
>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/TestScript.tcl
>>>  89/ 96 Testing Slicer3CLTest4
>>>
>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>> --eval puts\ testing\ ,.\ exit\ 0
>>>  90/ 96 Testing Slicer3CLTest5
>>>
>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>> --exec puts\ testing\ ,.\ exit\ 0
>>>  91/ 96 Testing Slicer3CLTest6
>>>
>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>> --script /home/ibanez/src/Slicer3/Applications/GUI/Testing/TestScript.tcl
>>>  92/ 96 Testing Slicer3CLTest7
>>>
>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>> --script
>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/LoadSceneTest.tcl
>>>  93/ 96 Testing Slicer3ScrollTest
>>>
>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>> --script
>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/ScrollTesting.tcl
>>>  94/ 96 Testing Slicer3MRMLUndo
>>>
>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>> --no-splash -f
>>> /home/ibanez/src/Slicer3-build/share/MRML/Testing/testUndo.tcl
>>>  95/ 96 Testing Slicer3MRMLVolume
>>>
>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>> --no-splash -f
>>> /home/ibanez/src/Slicer3-build/share/MRML/Testing/testVolume.tcl
>>>
>>>
>>> ---
>>>
>>> Also,
>>>
>>> Given the large amount of memory leaks currently
>>> present in Slicer3, I propose to engage in a campaign
>>> of vtkSmartPointer<>  adoption.
>>>
>>> Are there any reasons for not using vtkSmartPointers
>>> massively in Slicer ?
>>>
>>>
>>>     Thanks,
>>>
>>>
>>>          Luis
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------------
>>> On Tue, Dec 22, 2009 at 7:42 PM, Steve Pieper <[hidden email]>
>>> wrote:
>>>> Hi Luis -
>>>>
>>>> Many thanks for stepping up as the champion of testing.  I would also
>>>> like
>>>> to encourage everyone to take creating and fixing tests seriously.  It
>>>> will
>>>> make all of our lives easier!
>>>>
>>>> Regarding this simplest of all tests that you mentioned:
>>>>
>>>>> Could you please suggest what should be
>>>>> added to the script in order to make a clean
>>>>> return  ?
>>>>>
>>>>> We could replace the current code in:
>>>>>
>>>>> Applications/GUI/Testing/CMakeLists.txt:
>>>>>
>>>>> add_test(Slicer3CLTest5 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
>>>>> --exec "puts testing ,. exit 0")
>>>>>
>>>> This test simply lets slicer start up then exits cleanly.  This test is a
>>>> good way to see if the GUI startup up and shuts down without crashing or
>>>> leaking.  I don't think the test needs to be changed.  In fact, I'll bet
>>>> it
>>>> touches a lot of code that has to all work right for this to pass (module
>>>> discovery, GUI startup, scene creation and deletion...).
>>>>
>>>> The tests are all passing in the Slicer-3-4 branch (including this simple
>>>> one).  Things have gotten broken in the current trunk and need to be
>>>> fixed.
>>>>
>>>> -----
>>>>
>>>> Slicer developers may want to consider running the following handy
>>>> command
>>>> lines before checking in code:
>>>>
>>>>  ./Slicer3 --exec exit
>>>>
>>>> (this checks that all of slicer will start and exit cleanly)
>>>>
>>>>  ./Slicer3 --no-modules --exec exit
>>>>
>>>> (this will test just the base of slicer: no modules will be loaded or
>>>> built).
>>>>
>>>>  ./Slicer3 --ignore-module <ModuleName> --exec exit
>>>>
>>>> (this will specifically leave out a module to help identify where a leak
>>>>  or
>>>> crash is coming from.  Note that the module's GUI class constructor will
>>>> be
>>>> called, but not the BuildGUI method, so can still generate test failures
>>>> even if you "--ignore" it).
>>>>
>>>> With these you should be able to test
>>>>
>>>> -----
>>>>
>>>> Some notes on the testing script options for slicer (see the variants
>>>> from
>>>> --help below - these are used in many of the tests currently included in
>>>> slicer):
>>>>
>>>> * the standard tcl 'exit [code]' keyword is overridden.  Instead of
>>>> exiting
>>>> right away, slicer's exit will return the flow of control to after the
>>>> StartApplication call in Slicer3.cxx.  This allows the regular widget
>>>> tear
>>>> down sequence to happen.  The return code is passed back as the exit code
>>>> of
>>>> the process.
>>>>
>>>> * if a script is specified using one command line options, it will run in
>>>> the slicer interpreter (either the tcl or python interpreters, depending
>>>> on
>>>> the form of the argument).  The *eval* form of the argument avoids
>>>> building
>>>> the GUI, while the *exec* form builds the GUI and then runs the script.
>>>>
>>>> * the --test-mode argument disables any writing to the registry that
>>>> would
>>>> otherwise occur (i.e. the screen size and layout are not saved on exit)
>>>>
>>>>
>>>> Happy Holidays,
>>>> -Steve
>>>>
>>>>
>>>>
>>>>
>>>>  --evalpython <std::string>
>>>>    Like --exec, but doesn't load slicer first
>>>>
>>>>  --execpython <std::string>
>>>>    Some Python code to execute after slicer loads. (note: cannot specify
>>>>    scene after --exec)
>>>>
>>>>  -v <std::string>,  --eval <std::string>
>>>>    Like --exec, but doesn't load slicer first
>>>>
>>>>  -e <std::string>,  --exec <std::string>
>>>>    Some code to execute after slicer loads. (note: cannot specify scene
>>>>    after --exec) (note: use ,. instead of ; between tcl statements)
>>>>
>>>>  -p <std::string>,  --script <std::string>
>>>>    Script to execute after slicer loads
>>>>
>>>>  -f <std::string>,  --file <std::string>
>>>>    Script to execute instead of loading slicer
>>>>
>>>>
>>>>
_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
Reply | Threaded
Open this post in threaded view
|

Re: TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

Luis Ibanez
Hi Steve,

   Yes, let's get them to pass.     :-)

After some consultation with Kitware Gurus,
it turned out that the use of vtkDebugLeaks
is not the proper way of finding the offending
code.

In particular, the singletons will always be
included in the report.   :-/


So we are left with using Valgrind (and coffee).

My problem with using Valgrind is that Slicer
uses the launching mechanism, therefore the
executable that we start may not be the one
that actually have the leaks.

If I use the Slicer3 executable from Slicer-build,
Valgrind doesn't find the debugging symbols.
I'm forced then to use the executable in

           Slicer-build/bin/Slicer3

...but that's not the one we are launching
when we run the tests...


     Luis

-------------------------------------------------------------
On Wed, Dec 23, 2009 at 4:36 PM, Steve Pieper <[hidden email]> wrote:

> Hi Luis -
>
> Thanks for the new tests - now let's get them passing :)
>
> Yes, we alway use the --test-mode flag on the cmake-based tests.  It skips
> the splash screen and also doesn't save any changes to the registry (and
> perhaps a few other things).
>
> Here's what I get on my build:
>
>> % ./Slicer3 --no-modules --exec exit
>> Setting up launch environment...
>> Starting Slicer: /Users/pieper/slicer3/latest/Slicer3-build
>> vtkDebugLeaks has detected LEAKS!
>> Class "vtkOutputWindow" has 1 instance still around.
>> Class "vtkEventBroker" has 1 instance still around.
>> Class "vtkFreeTypeUtilities" has 1 instance still around.
>> Class "vtkStringArray" has 2 instances still around.
>> Class "vtkTimerLog" has 1 instance still around.
>>
>> There are memory leaks
>> child process exited abnormally
>>
>
> The vtkFreeTypeUtilities class is odd - perhaps it is created by the
> vtkOutputWindow.  The vtkStringArray and vtkTimerLog might also be
> associated with this.
>
> JC recently introduced a new singleton pattern for the vtkEventBroker so I'm
> not sure why that is leaking.
>
> Typically even a single missing Delete() can lead to a whole cascade of
> leaks.  Tracking them down is a bit of a puzzle.  Satisfying when they get
> cleared up, but frustrating that they keep re-appearing.  Perhaps the
> vtkSmartPointer conversion isn't such a bad idea...
>
> -Steve
>
> Luis Ibanez wrote:
>>
>> On Wed, Dec 23, 2009 at 2:06 PM, Steve Pieper <[hidden email]>
>> wrote:
>>>
>>> Hi Luis -
>>>
>>> Yes, it would make a lot of sense to add the following as a test:
>>>
>>>  ./Slicer3 --no-modules --exec exit
>>>
>>
>> The test has been added
>>
>> http://viewvc.slicer.org/viewcvs.cgi/trunk/Applications/GUI/Testing/CMakeLists.txt?rev=11378&sortby=date&r2=11378&r1=11255
>>
>> They are:
>>
>> * Slicer3CLTest8
>> * Slicer3CLTest9
>>
>> add_test(Slicer3CLTest8 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
>> --no-modules)
>> add_test(Slicer3CLTest9 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
>> --no-modules --exec "exit")
>>
>> As you pointed out, both are failing
>>
>> BTW,
>> I assume that we also need the "--test-mode" flag,
>> is that correct ?
>>
>>
>>> On the current trunk, this test fails :(  No doubt as a result of some
>>> recent checkins.  Having this checked automatically will help (assuming
>>> people pay attention when the test stops working!!!).
>>>
>>
>> Could you provide any guess as to how recent the
>> offending commit may have been ?
>>
>> Cleaning up the Dashboard is
>> a critical task at this point...
>>
>>
>>> Also, it would be great to add a set of tests, one for each module that
>>> is
>>> bundled with the slicer build of the form:
>>>
>>>  ./Slicer3 --ignore-module <ModuleName> --exec exit
>>>
>>> The other command line arguments would also be great to test.
>>>
>>>
>>> Regarding vtkSmartPointer in slicer, Jim has written some excellent
>>> guidelines for how it should be used.
>>>
>>> http://www.slicer.org/slicerWiki/index.php/Slicer3:Memory_Management
>>>
>>> The primary development of slicer has been to copy the style of vtk for
>>> consistency and the New()/Delete() pattern is widely used inside vtk.
>>> Making
>>> a change to smart pointers would be a lot of work and probably not the
>>> most
>>> efficient way to fix the current memory leaks.  But if someone wants to
>>> take
>>> this on (and does it well) I wouldn't object.
>>>
>>> Best,
>>> Steve
>>>
>>>
>>>
>>> Luis Ibanez wrote:
>>>>
>>>> Hi Steve,
>>>>
>>>> Thanks a lot for providing more guidelines for testing,
>>>>
>>>> I was actually wondering a moment ago if the instructions at
>>>> http://www.slicer.org/slicerWiki/index.php/Slicer3:Testing
>>>> were still valid.
>>>>
>>>> It seems that some of these suggested tests
>>>> are the ones executed  by the tests:
>>>>
>>>>   ctest -R Slicer3CLTest -N
>>>>
>>>>  86/ 96 Testing Slicer3CLTest1
>>>>  87/ 96 Testing Slicer3CLTest2
>>>>  88/ 96 Testing Slicer3CLTest3
>>>>  89/ 96 Testing Slicer3CLTest4
>>>>  90/ 96 Testing Slicer3CLTest5
>>>>  91/ 96 Testing Slicer3CLTest6
>>>>  92/ 96 Testing Slicer3CLTest7
>>>>
>>>> But.. it also seems that we have not all of the
>>>> ones suggested in
>>>> http://www.slicer.org/slicerWiki/index.php/Slicer3:Testing
>>>> and not all the ones in your email.
>>>>
>>>> Shouldn't we add the missing ones ?
>>>>
>>>>
>>>> The current ones are:
>>>>
>>>>   ctest -R Slicer3CLTest -N -V
>>>>
>>>>
>>>> 86/ 96 Testing Slicer3CLTest1
>>>>
>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode --help
>>>>  87/ 96 Testing Slicer3CLTest2
>>>>
>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --xml --test-mode
>>>>  88/ 96 Testing Slicer3CLTest3
>>>>
>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode -f
>>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/TestScript.tcl
>>>>  89/ 96 Testing Slicer3CLTest4
>>>>
>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>> --eval puts\ testing\ ,.\ exit\ 0
>>>>  90/ 96 Testing Slicer3CLTest5
>>>>
>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>> --exec puts\ testing\ ,.\ exit\ 0
>>>>  91/ 96 Testing Slicer3CLTest6
>>>>
>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>> --script
>>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/TestScript.tcl
>>>>  92/ 96 Testing Slicer3CLTest7
>>>>
>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>> --script
>>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/LoadSceneTest.tcl
>>>>  93/ 96 Testing Slicer3ScrollTest
>>>>
>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>> --script
>>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/ScrollTesting.tcl
>>>>  94/ 96 Testing Slicer3MRMLUndo
>>>>
>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>> --no-splash -f
>>>> /home/ibanez/src/Slicer3-build/share/MRML/Testing/testUndo.tcl
>>>>  95/ 96 Testing Slicer3MRMLVolume
>>>>
>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>> --no-splash -f
>>>> /home/ibanez/src/Slicer3-build/share/MRML/Testing/testVolume.tcl
>>>>
>>>>
>>>> ---
>>>>
>>>> Also,
>>>>
>>>> Given the large amount of memory leaks currently
>>>> present in Slicer3, I propose to engage in a campaign
>>>> of vtkSmartPointer<>  adoption.
>>>>
>>>> Are there any reasons for not using vtkSmartPointers
>>>> massively in Slicer ?
>>>>
>>>>
>>>>    Thanks,
>>>>
>>>>
>>>>         Luis
>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------------
>>>> On Tue, Dec 22, 2009 at 7:42 PM, Steve Pieper <[hidden email]>
>>>> wrote:
>>>>>
>>>>> Hi Luis -
>>>>>
>>>>> Many thanks for stepping up as the champion of testing.  I would also
>>>>> like
>>>>> to encourage everyone to take creating and fixing tests seriously.  It
>>>>> will
>>>>> make all of our lives easier!
>>>>>
>>>>> Regarding this simplest of all tests that you mentioned:
>>>>>
>>>>>> Could you please suggest what should be
>>>>>> added to the script in order to make a clean
>>>>>> return  ?
>>>>>>
>>>>>> We could replace the current code in:
>>>>>>
>>>>>> Applications/GUI/Testing/CMakeLists.txt:
>>>>>>
>>>>>> add_test(Slicer3CLTest5 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
>>>>>> --exec "puts testing ,. exit 0")
>>>>>>
>>>>> This test simply lets slicer start up then exits cleanly.  This test is
>>>>> a
>>>>> good way to see if the GUI startup up and shuts down without crashing
>>>>> or
>>>>> leaking.  I don't think the test needs to be changed.  In fact, I'll
>>>>> bet
>>>>> it
>>>>> touches a lot of code that has to all work right for this to pass
>>>>> (module
>>>>> discovery, GUI startup, scene creation and deletion...).
>>>>>
>>>>> The tests are all passing in the Slicer-3-4 branch (including this
>>>>> simple
>>>>> one).  Things have gotten broken in the current trunk and need to be
>>>>> fixed.
>>>>>
>>>>> -----
>>>>>
>>>>> Slicer developers may want to consider running the following handy
>>>>> command
>>>>> lines before checking in code:
>>>>>
>>>>>  ./Slicer3 --exec exit
>>>>>
>>>>> (this checks that all of slicer will start and exit cleanly)
>>>>>
>>>>>  ./Slicer3 --no-modules --exec exit
>>>>>
>>>>> (this will test just the base of slicer: no modules will be loaded or
>>>>> built).
>>>>>
>>>>>  ./Slicer3 --ignore-module <ModuleName> --exec exit
>>>>>
>>>>> (this will specifically leave out a module to help identify where a
>>>>> leak
>>>>>  or
>>>>> crash is coming from.  Note that the module's GUI class constructor
>>>>> will
>>>>> be
>>>>> called, but not the BuildGUI method, so can still generate test
>>>>> failures
>>>>> even if you "--ignore" it).
>>>>>
>>>>> With these you should be able to test
>>>>>
>>>>> -----
>>>>>
>>>>> Some notes on the testing script options for slicer (see the variants
>>>>> from
>>>>> --help below - these are used in many of the tests currently included
>>>>> in
>>>>> slicer):
>>>>>
>>>>> * the standard tcl 'exit [code]' keyword is overridden.  Instead of
>>>>> exiting
>>>>> right away, slicer's exit will return the flow of control to after the
>>>>> StartApplication call in Slicer3.cxx.  This allows the regular widget
>>>>> tear
>>>>> down sequence to happen.  The return code is passed back as the exit
>>>>> code
>>>>> of
>>>>> the process.
>>>>>
>>>>> * if a script is specified using one command line options, it will run
>>>>> in
>>>>> the slicer interpreter (either the tcl or python interpreters,
>>>>> depending
>>>>> on
>>>>> the form of the argument).  The *eval* form of the argument avoids
>>>>> building
>>>>> the GUI, while the *exec* form builds the GUI and then runs the script.
>>>>>
>>>>> * the --test-mode argument disables any writing to the registry that
>>>>> would
>>>>> otherwise occur (i.e. the screen size and layout are not saved on exit)
>>>>>
>>>>>
>>>>> Happy Holidays,
>>>>> -Steve
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>  --evalpython <std::string>
>>>>>   Like --exec, but doesn't load slicer first
>>>>>
>>>>>  --execpython <std::string>
>>>>>   Some Python code to execute after slicer loads. (note: cannot specify
>>>>>   scene after --exec)
>>>>>
>>>>>  -v <std::string>,  --eval <std::string>
>>>>>   Like --exec, but doesn't load slicer first
>>>>>
>>>>>  -e <std::string>,  --exec <std::string>
>>>>>   Some code to execute after slicer loads. (note: cannot specify scene
>>>>>   after --exec) (note: use ,. instead of ; between tcl statements)
>>>>>
>>>>>  -p <std::string>,  --script <std::string>
>>>>>   Script to execute after slicer loads
>>>>>
>>>>>  -f <std::string>,  --file <std::string>
>>>>>   Script to execute instead of loading slicer
>>>>>
>>>>>
>>>>>
>
_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
Reply | Threaded
Open this post in threaded view
|

Re: TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

pieper
Administrator
Hi Luis -

Regarding singletons: until the pattern was changed very recently -- by
a Kitware guru ;) -- we were always able to use vtkDebugLeaks to find
the offending code.  If that is no longer possible then perhaps we
should revert to the previous singleton pattern which worked well for
years and was very useful.

Regarding valgrind, I have used it quite a bit with slicer and is useful
in it's own way (not by itself, but with coffee-enhanced human effort!).

The command line is in a comment in this file:

CMake/Slicer3ValgrindSuppressions.supp

As you mentioned, one needs to use bin/Slicer3-real as the entry point
for valgrind. It also helps to use the suppression files.  There are
many false positives from valgrind, which are mostly caught by the
supression files (but not completely).

-Steve



Luis Ibanez wrote:

> Hi Steve,
>
>    Yes, let's get them to pass.     :-)
>
> After some consultation with Kitware Gurus,
> it turned out that the use of vtkDebugLeaks
> is not the proper way of finding the offending
> code.
>
> In particular, the singletons will always be
> included in the report.   :-/
>
>
> So we are left with using Valgrind (and coffee).
>
> My problem with using Valgrind is that Slicer
> uses the launching mechanism, therefore the
> executable that we start may not be the one
> that actually have the leaks.
>
> If I use the Slicer3 executable from Slicer-build,
> Valgrind doesn't find the debugging symbols.
> I'm forced then to use the executable in
>
>            Slicer-build/bin/Slicer3
>
> ...but that's not the one we are launching
> when we run the tests...
>
>
>      Luis
>
> -------------------------------------------------------------
> On Wed, Dec 23, 2009 at 4:36 PM, Steve Pieper <[hidden email]> wrote:
>> Hi Luis -
>>
>> Thanks for the new tests - now let's get them passing :)
>>
>> Yes, we alway use the --test-mode flag on the cmake-based tests.  It skips
>> the splash screen and also doesn't save any changes to the registry (and
>> perhaps a few other things).
>>
>> Here's what I get on my build:
>>
>>> % ./Slicer3 --no-modules --exec exit
>>> Setting up launch environment...
>>> Starting Slicer: /Users/pieper/slicer3/latest/Slicer3-build
>>> vtkDebugLeaks has detected LEAKS!
>>> Class "vtkOutputWindow" has 1 instance still around.
>>> Class "vtkEventBroker" has 1 instance still around.
>>> Class "vtkFreeTypeUtilities" has 1 instance still around.
>>> Class "vtkStringArray" has 2 instances still around.
>>> Class "vtkTimerLog" has 1 instance still around.
>>>
>>> There are memory leaks
>>> child process exited abnormally
>>>
>> The vtkFreeTypeUtilities class is odd - perhaps it is created by the
>> vtkOutputWindow.  The vtkStringArray and vtkTimerLog might also be
>> associated with this.
>>
>> JC recently introduced a new singleton pattern for the vtkEventBroker so I'm
>> not sure why that is leaking.
>>
>> Typically even a single missing Delete() can lead to a whole cascade of
>> leaks.  Tracking them down is a bit of a puzzle.  Satisfying when they get
>> cleared up, but frustrating that they keep re-appearing.  Perhaps the
>> vtkSmartPointer conversion isn't such a bad idea...
>>
>> -Steve
>>
>> Luis Ibanez wrote:
>>> On Wed, Dec 23, 2009 at 2:06 PM, Steve Pieper <[hidden email]>
>>> wrote:
>>>> Hi Luis -
>>>>
>>>> Yes, it would make a lot of sense to add the following as a test:
>>>>
>>>>  ./Slicer3 --no-modules --exec exit
>>>>
>>> The test has been added
>>>
>>> http://viewvc.slicer.org/viewcvs.cgi/trunk/Applications/GUI/Testing/CMakeLists.txt?rev=11378&sortby=date&r2=11378&r1=11255
>>>
>>> They are:
>>>
>>> * Slicer3CLTest8
>>> * Slicer3CLTest9
>>>
>>> add_test(Slicer3CLTest8 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
>>> --no-modules)
>>> add_test(Slicer3CLTest9 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
>>> --no-modules --exec "exit")
>>>
>>> As you pointed out, both are failing
>>>
>>> BTW,
>>> I assume that we also need the "--test-mode" flag,
>>> is that correct ?
>>>
>>>
>>>> On the current trunk, this test fails :(  No doubt as a result of some
>>>> recent checkins.  Having this checked automatically will help (assuming
>>>> people pay attention when the test stops working!!!).
>>>>
>>> Could you provide any guess as to how recent the
>>> offending commit may have been ?
>>>
>>> Cleaning up the Dashboard is
>>> a critical task at this point...
>>>
>>>
>>>> Also, it would be great to add a set of tests, one for each module that
>>>> is
>>>> bundled with the slicer build of the form:
>>>>
>>>>  ./Slicer3 --ignore-module <ModuleName> --exec exit
>>>>
>>>> The other command line arguments would also be great to test.
>>>>
>>>>
>>>> Regarding vtkSmartPointer in slicer, Jim has written some excellent
>>>> guidelines for how it should be used.
>>>>
>>>> http://www.slicer.org/slicerWiki/index.php/Slicer3:Memory_Management
>>>>
>>>> The primary development of slicer has been to copy the style of vtk for
>>>> consistency and the New()/Delete() pattern is widely used inside vtk.
>>>> Making
>>>> a change to smart pointers would be a lot of work and probably not the
>>>> most
>>>> efficient way to fix the current memory leaks.  But if someone wants to
>>>> take
>>>> this on (and does it well) I wouldn't object.
>>>>
>>>> Best,
>>>> Steve
>>>>
>>>>
>>>>
>>>> Luis Ibanez wrote:
>>>>> Hi Steve,
>>>>>
>>>>> Thanks a lot for providing more guidelines for testing,
>>>>>
>>>>> I was actually wondering a moment ago if the instructions at
>>>>> http://www.slicer.org/slicerWiki/index.php/Slicer3:Testing
>>>>> were still valid.
>>>>>
>>>>> It seems that some of these suggested tests
>>>>> are the ones executed  by the tests:
>>>>>
>>>>>   ctest -R Slicer3CLTest -N
>>>>>
>>>>>  86/ 96 Testing Slicer3CLTest1
>>>>>  87/ 96 Testing Slicer3CLTest2
>>>>>  88/ 96 Testing Slicer3CLTest3
>>>>>  89/ 96 Testing Slicer3CLTest4
>>>>>  90/ 96 Testing Slicer3CLTest5
>>>>>  91/ 96 Testing Slicer3CLTest6
>>>>>  92/ 96 Testing Slicer3CLTest7
>>>>>
>>>>> But.. it also seems that we have not all of the
>>>>> ones suggested in
>>>>> http://www.slicer.org/slicerWiki/index.php/Slicer3:Testing
>>>>> and not all the ones in your email.
>>>>>
>>>>> Shouldn't we add the missing ones ?
>>>>>
>>>>>
>>>>> The current ones are:
>>>>>
>>>>>   ctest -R Slicer3CLTest -N -V
>>>>>
>>>>>
>>>>> 86/ 96 Testing Slicer3CLTest1
>>>>>
>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode --help
>>>>>  87/ 96 Testing Slicer3CLTest2
>>>>>
>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --xml --test-mode
>>>>>  88/ 96 Testing Slicer3CLTest3
>>>>>
>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode -f
>>>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/TestScript.tcl
>>>>>  89/ 96 Testing Slicer3CLTest4
>>>>>
>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>> --eval puts\ testing\ ,.\ exit\ 0
>>>>>  90/ 96 Testing Slicer3CLTest5
>>>>>
>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>> --exec puts\ testing\ ,.\ exit\ 0
>>>>>  91/ 96 Testing Slicer3CLTest6
>>>>>
>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>> --script
>>>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/TestScript.tcl
>>>>>  92/ 96 Testing Slicer3CLTest7
>>>>>
>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>> --script
>>>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/LoadSceneTest.tcl
>>>>>  93/ 96 Testing Slicer3ScrollTest
>>>>>
>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>> --script
>>>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/ScrollTesting.tcl
>>>>>  94/ 96 Testing Slicer3MRMLUndo
>>>>>
>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>> --no-splash -f
>>>>> /home/ibanez/src/Slicer3-build/share/MRML/Testing/testUndo.tcl
>>>>>  95/ 96 Testing Slicer3MRMLVolume
>>>>>
>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>> --no-splash -f
>>>>> /home/ibanez/src/Slicer3-build/share/MRML/Testing/testVolume.tcl
>>>>>
>>>>>
>>>>> ---
>>>>>
>>>>> Also,
>>>>>
>>>>> Given the large amount of memory leaks currently
>>>>> present in Slicer3, I propose to engage in a campaign
>>>>> of vtkSmartPointer<>  adoption.
>>>>>
>>>>> Are there any reasons for not using vtkSmartPointers
>>>>> massively in Slicer ?
>>>>>
>>>>>
>>>>>    Thanks,
>>>>>
>>>>>
>>>>>         Luis
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------------
>>>>> On Tue, Dec 22, 2009 at 7:42 PM, Steve Pieper <[hidden email]>
>>>>> wrote:
>>>>>> Hi Luis -
>>>>>>
>>>>>> Many thanks for stepping up as the champion of testing.  I would also
>>>>>> like
>>>>>> to encourage everyone to take creating and fixing tests seriously.  It
>>>>>> will
>>>>>> make all of our lives easier!
>>>>>>
>>>>>> Regarding this simplest of all tests that you mentioned:
>>>>>>
>>>>>>> Could you please suggest what should be
>>>>>>> added to the script in order to make a clean
>>>>>>> return  ?
>>>>>>>
>>>>>>> We could replace the current code in:
>>>>>>>
>>>>>>> Applications/GUI/Testing/CMakeLists.txt:
>>>>>>>
>>>>>>> add_test(Slicer3CLTest5 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
>>>>>>> --exec "puts testing ,. exit 0")
>>>>>>>
>>>>>> This test simply lets slicer start up then exits cleanly.  This test is
>>>>>> a
>>>>>> good way to see if the GUI startup up and shuts down without crashing
>>>>>> or
>>>>>> leaking.  I don't think the test needs to be changed.  In fact, I'll
>>>>>> bet
>>>>>> it
>>>>>> touches a lot of code that has to all work right for this to pass
>>>>>> (module
>>>>>> discovery, GUI startup, scene creation and deletion...).
>>>>>>
>>>>>> The tests are all passing in the Slicer-3-4 branch (including this
>>>>>> simple
>>>>>> one).  Things have gotten broken in the current trunk and need to be
>>>>>> fixed.
>>>>>>
>>>>>> -----
>>>>>>
>>>>>> Slicer developers may want to consider running the following handy
>>>>>> command
>>>>>> lines before checking in code:
>>>>>>
>>>>>>  ./Slicer3 --exec exit
>>>>>>
>>>>>> (this checks that all of slicer will start and exit cleanly)
>>>>>>
>>>>>>  ./Slicer3 --no-modules --exec exit
>>>>>>
>>>>>> (this will test just the base of slicer: no modules will be loaded or
>>>>>> built).
>>>>>>
>>>>>>  ./Slicer3 --ignore-module <ModuleName> --exec exit
>>>>>>
>>>>>> (this will specifically leave out a module to help identify where a
>>>>>> leak
>>>>>>  or
>>>>>> crash is coming from.  Note that the module's GUI class constructor
>>>>>> will
>>>>>> be
>>>>>> called, but not the BuildGUI method, so can still generate test
>>>>>> failures
>>>>>> even if you "--ignore" it).
>>>>>>
>>>>>> With these you should be able to test
>>>>>>
>>>>>> -----
>>>>>>
>>>>>> Some notes on the testing script options for slicer (see the variants
>>>>>> from
>>>>>> --help below - these are used in many of the tests currently included
>>>>>> in
>>>>>> slicer):
>>>>>>
>>>>>> * the standard tcl 'exit [code]' keyword is overridden.  Instead of
>>>>>> exiting
>>>>>> right away, slicer's exit will return the flow of control to after the
>>>>>> StartApplication call in Slicer3.cxx.  This allows the regular widget
>>>>>> tear
>>>>>> down sequence to happen.  The return code is passed back as the exit
>>>>>> code
>>>>>> of
>>>>>> the process.
>>>>>>
>>>>>> * if a script is specified using one command line options, it will run
>>>>>> in
>>>>>> the slicer interpreter (either the tcl or python interpreters,
>>>>>> depending
>>>>>> on
>>>>>> the form of the argument).  The *eval* form of the argument avoids
>>>>>> building
>>>>>> the GUI, while the *exec* form builds the GUI and then runs the script.
>>>>>>
>>>>>> * the --test-mode argument disables any writing to the registry that
>>>>>> would
>>>>>> otherwise occur (i.e. the screen size and layout are not saved on exit)
>>>>>>
>>>>>>
>>>>>> Happy Holidays,
>>>>>> -Steve
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>  --evalpython <std::string>
>>>>>>   Like --exec, but doesn't load slicer first
>>>>>>
>>>>>>  --execpython <std::string>
>>>>>>   Some Python code to execute after slicer loads. (note: cannot specify
>>>>>>   scene after --exec)
>>>>>>
>>>>>>  -v <std::string>,  --eval <std::string>
>>>>>>   Like --exec, but doesn't load slicer first
>>>>>>
>>>>>>  -e <std::string>,  --exec <std::string>
>>>>>>   Some code to execute after slicer loads. (note: cannot specify scene
>>>>>>   after --exec) (note: use ,. instead of ; between tcl statements)
>>>>>>
>>>>>>  -p <std::string>,  --script <std::string>
>>>>>>   Script to execute after slicer loads
>>>>>>
>>>>>>  -f <std::string>,  --file <std::string>
>>>>>>   Script to execute instead of loading slicer
>>>>>>
>>>>>>
>>>>>>
_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
Reply | Threaded
Open this post in threaded view
|

Re: TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

Luis Ibanez
Hi Steve,

Thanks, this is very helpful.

I'll keep looking at resolving the leaks
of the Slicer3CLTest5.


    Luis


-------------------------------------------
On Wed, Dec 23, 2009 at 8:23 PM, Steve Pieper <[hidden email]> wrote:

> Hi Luis -
>
> Regarding singletons: until the pattern was changed very recently -- by a
> Kitware guru ;) -- we were always able to use vtkDebugLeaks to find the
> offending code.  If that is no longer possible then perhaps we should revert
> to the previous singleton pattern which worked well for years and was very
> useful.
>
> Regarding valgrind, I have used it quite a bit with slicer and is useful in
> it's own way (not by itself, but with coffee-enhanced human effort!).
>
> The command line is in a comment in this file:
>
> CMake/Slicer3ValgrindSuppressions.supp
>
> As you mentioned, one needs to use bin/Slicer3-real as the entry point for
> valgrind. It also helps to use the suppression files.  There are many false
> positives from valgrind, which are mostly caught by the supression files
> (but not completely).
>
> -Steve
>
>
>
> Luis Ibanez wrote:
>>
>> Hi Steve,
>>
>>   Yes, let's get them to pass.     :-)
>>
>> After some consultation with Kitware Gurus,
>> it turned out that the use of vtkDebugLeaks
>> is not the proper way of finding the offending
>> code.
>>
>> In particular, the singletons will always be
>> included in the report.   :-/
>>
>>
>> So we are left with using Valgrind (and coffee).
>>
>> My problem with using Valgrind is that Slicer
>> uses the launching mechanism, therefore the
>> executable that we start may not be the one
>> that actually have the leaks.
>>
>> If I use the Slicer3 executable from Slicer-build,
>> Valgrind doesn't find the debugging symbols.
>> I'm forced then to use the executable in
>>
>>           Slicer-build/bin/Slicer3
>>
>> ...but that's not the one we are launching
>> when we run the tests...
>>
>>
>>     Luis
>>
>> -------------------------------------------------------------
>> On Wed, Dec 23, 2009 at 4:36 PM, Steve Pieper <[hidden email]>
>> wrote:
>>>
>>> Hi Luis -
>>>
>>> Thanks for the new tests - now let's get them passing :)
>>>
>>> Yes, we alway use the --test-mode flag on the cmake-based tests.  It
>>> skips
>>> the splash screen and also doesn't save any changes to the registry (and
>>> perhaps a few other things).
>>>
>>> Here's what I get on my build:
>>>
>>>> % ./Slicer3 --no-modules --exec exit
>>>> Setting up launch environment...
>>>> Starting Slicer: /Users/pieper/slicer3/latest/Slicer3-build
>>>> vtkDebugLeaks has detected LEAKS!
>>>> Class "vtkOutputWindow" has 1 instance still around.
>>>> Class "vtkEventBroker" has 1 instance still around.
>>>> Class "vtkFreeTypeUtilities" has 1 instance still around.
>>>> Class "vtkStringArray" has 2 instances still around.
>>>> Class "vtkTimerLog" has 1 instance still around.
>>>>
>>>> There are memory leaks
>>>> child process exited abnormally
>>>>
>>> The vtkFreeTypeUtilities class is odd - perhaps it is created by the
>>> vtkOutputWindow.  The vtkStringArray and vtkTimerLog might also be
>>> associated with this.
>>>
>>> JC recently introduced a new singleton pattern for the vtkEventBroker so
>>> I'm
>>> not sure why that is leaking.
>>>
>>> Typically even a single missing Delete() can lead to a whole cascade of
>>> leaks.  Tracking them down is a bit of a puzzle.  Satisfying when they
>>> get
>>> cleared up, but frustrating that they keep re-appearing.  Perhaps the
>>> vtkSmartPointer conversion isn't such a bad idea...
>>>
>>> -Steve
>>>
>>> Luis Ibanez wrote:
>>>>
>>>> On Wed, Dec 23, 2009 at 2:06 PM, Steve Pieper <[hidden email]>
>>>> wrote:
>>>>>
>>>>> Hi Luis -
>>>>>
>>>>> Yes, it would make a lot of sense to add the following as a test:
>>>>>
>>>>>  ./Slicer3 --no-modules --exec exit
>>>>>
>>>> The test has been added
>>>>
>>>>
>>>> http://viewvc.slicer.org/viewcvs.cgi/trunk/Applications/GUI/Testing/CMakeLists.txt?rev=11378&sortby=date&r2=11378&r1=11255
>>>>
>>>> They are:
>>>>
>>>> * Slicer3CLTest8
>>>> * Slicer3CLTest9
>>>>
>>>> add_test(Slicer3CLTest8 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
>>>> --no-modules)
>>>> add_test(Slicer3CLTest9 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
>>>> --no-modules --exec "exit")
>>>>
>>>> As you pointed out, both are failing
>>>>
>>>> BTW,
>>>> I assume that we also need the "--test-mode" flag,
>>>> is that correct ?
>>>>
>>>>
>>>>> On the current trunk, this test fails :(  No doubt as a result of some
>>>>> recent checkins.  Having this checked automatically will help (assuming
>>>>> people pay attention when the test stops working!!!).
>>>>>
>>>> Could you provide any guess as to how recent the
>>>> offending commit may have been ?
>>>>
>>>> Cleaning up the Dashboard is
>>>> a critical task at this point...
>>>>
>>>>
>>>>> Also, it would be great to add a set of tests, one for each module that
>>>>> is
>>>>> bundled with the slicer build of the form:
>>>>>
>>>>>  ./Slicer3 --ignore-module <ModuleName> --exec exit
>>>>>
>>>>> The other command line arguments would also be great to test.
>>>>>
>>>>>
>>>>> Regarding vtkSmartPointer in slicer, Jim has written some excellent
>>>>> guidelines for how it should be used.
>>>>>
>>>>> http://www.slicer.org/slicerWiki/index.php/Slicer3:Memory_Management
>>>>>
>>>>> The primary development of slicer has been to copy the style of vtk for
>>>>> consistency and the New()/Delete() pattern is widely used inside vtk.
>>>>> Making
>>>>> a change to smart pointers would be a lot of work and probably not the
>>>>> most
>>>>> efficient way to fix the current memory leaks.  But if someone wants to
>>>>> take
>>>>> this on (and does it well) I wouldn't object.
>>>>>
>>>>> Best,
>>>>> Steve
>>>>>
>>>>>
>>>>>
>>>>> Luis Ibanez wrote:
>>>>>>
>>>>>> Hi Steve,
>>>>>>
>>>>>> Thanks a lot for providing more guidelines for testing,
>>>>>>
>>>>>> I was actually wondering a moment ago if the instructions at
>>>>>> http://www.slicer.org/slicerWiki/index.php/Slicer3:Testing
>>>>>> were still valid.
>>>>>>
>>>>>> It seems that some of these suggested tests
>>>>>> are the ones executed  by the tests:
>>>>>>
>>>>>>  ctest -R Slicer3CLTest -N
>>>>>>
>>>>>>  86/ 96 Testing Slicer3CLTest1
>>>>>>  87/ 96 Testing Slicer3CLTest2
>>>>>>  88/ 96 Testing Slicer3CLTest3
>>>>>>  89/ 96 Testing Slicer3CLTest4
>>>>>>  90/ 96 Testing Slicer3CLTest5
>>>>>>  91/ 96 Testing Slicer3CLTest6
>>>>>>  92/ 96 Testing Slicer3CLTest7
>>>>>>
>>>>>> But.. it also seems that we have not all of the
>>>>>> ones suggested in
>>>>>> http://www.slicer.org/slicerWiki/index.php/Slicer3:Testing
>>>>>> and not all the ones in your email.
>>>>>>
>>>>>> Shouldn't we add the missing ones ?
>>>>>>
>>>>>>
>>>>>> The current ones are:
>>>>>>
>>>>>>  ctest -R Slicer3CLTest -N -V
>>>>>>
>>>>>>
>>>>>> 86/ 96 Testing Slicer3CLTest1
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>>> --help
>>>>>>  87/ 96 Testing Slicer3CLTest2
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --xml --test-mode
>>>>>>  88/ 96 Testing Slicer3CLTest3
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode -f
>>>>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/TestScript.tcl
>>>>>>  89/ 96 Testing Slicer3CLTest4
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>>> --eval puts\ testing\ ,.\ exit\ 0
>>>>>>  90/ 96 Testing Slicer3CLTest5
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>>> --exec puts\ testing\ ,.\ exit\ 0
>>>>>>  91/ 96 Testing Slicer3CLTest6
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>>> --script
>>>>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/TestScript.tcl
>>>>>>  92/ 96 Testing Slicer3CLTest7
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>>> --script
>>>>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/LoadSceneTest.tcl
>>>>>>  93/ 96 Testing Slicer3ScrollTest
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>>> --script
>>>>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/ScrollTesting.tcl
>>>>>>  94/ 96 Testing Slicer3MRMLUndo
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>>> --no-splash -f
>>>>>> /home/ibanez/src/Slicer3-build/share/MRML/Testing/testUndo.tcl
>>>>>>  95/ 96 Testing Slicer3MRMLVolume
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>>> --no-splash -f
>>>>>> /home/ibanez/src/Slicer3-build/share/MRML/Testing/testVolume.tcl
>>>>>>
>>>>>>
>>>>>> ---
>>>>>>
>>>>>> Also,
>>>>>>
>>>>>> Given the large amount of memory leaks currently
>>>>>> present in Slicer3, I propose to engage in a campaign
>>>>>> of vtkSmartPointer<>  adoption.
>>>>>>
>>>>>> Are there any reasons for not using vtkSmartPointers
>>>>>> massively in Slicer ?
>>>>>>
>>>>>>
>>>>>>   Thanks,
>>>>>>
>>>>>>
>>>>>>        Luis
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------------
>>>>>> On Tue, Dec 22, 2009 at 7:42 PM, Steve Pieper <[hidden email]>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi Luis -
>>>>>>>
>>>>>>> Many thanks for stepping up as the champion of testing.  I would also
>>>>>>> like
>>>>>>> to encourage everyone to take creating and fixing tests seriously.
>>>>>>>  It
>>>>>>> will
>>>>>>> make all of our lives easier!
>>>>>>>
>>>>>>> Regarding this simplest of all tests that you mentioned:
>>>>>>>
>>>>>>>> Could you please suggest what should be
>>>>>>>> added to the script in order to make a clean
>>>>>>>> return  ?
>>>>>>>>
>>>>>>>> We could replace the current code in:
>>>>>>>>
>>>>>>>> Applications/GUI/Testing/CMakeLists.txt:
>>>>>>>>
>>>>>>>> add_test(Slicer3CLTest5 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
>>>>>>>> --exec "puts testing ,. exit 0")
>>>>>>>>
>>>>>>> This test simply lets slicer start up then exits cleanly.  This test
>>>>>>> is
>>>>>>> a
>>>>>>> good way to see if the GUI startup up and shuts down without crashing
>>>>>>> or
>>>>>>> leaking.  I don't think the test needs to be changed.  In fact, I'll
>>>>>>> bet
>>>>>>> it
>>>>>>> touches a lot of code that has to all work right for this to pass
>>>>>>> (module
>>>>>>> discovery, GUI startup, scene creation and deletion...).
>>>>>>>
>>>>>>> The tests are all passing in the Slicer-3-4 branch (including this
>>>>>>> simple
>>>>>>> one).  Things have gotten broken in the current trunk and need to be
>>>>>>> fixed.
>>>>>>>
>>>>>>> -----
>>>>>>>
>>>>>>> Slicer developers may want to consider running the following handy
>>>>>>> command
>>>>>>> lines before checking in code:
>>>>>>>
>>>>>>>  ./Slicer3 --exec exit
>>>>>>>
>>>>>>> (this checks that all of slicer will start and exit cleanly)
>>>>>>>
>>>>>>>  ./Slicer3 --no-modules --exec exit
>>>>>>>
>>>>>>> (this will test just the base of slicer: no modules will be loaded or
>>>>>>> built).
>>>>>>>
>>>>>>>  ./Slicer3 --ignore-module <ModuleName> --exec exit
>>>>>>>
>>>>>>> (this will specifically leave out a module to help identify where a
>>>>>>> leak
>>>>>>>  or
>>>>>>> crash is coming from.  Note that the module's GUI class constructor
>>>>>>> will
>>>>>>> be
>>>>>>> called, but not the BuildGUI method, so can still generate test
>>>>>>> failures
>>>>>>> even if you "--ignore" it).
>>>>>>>
>>>>>>> With these you should be able to test
>>>>>>>
>>>>>>> -----
>>>>>>>
>>>>>>> Some notes on the testing script options for slicer (see the variants
>>>>>>> from
>>>>>>> --help below - these are used in many of the tests currently included
>>>>>>> in
>>>>>>> slicer):
>>>>>>>
>>>>>>> * the standard tcl 'exit [code]' keyword is overridden.  Instead of
>>>>>>> exiting
>>>>>>> right away, slicer's exit will return the flow of control to after
>>>>>>> the
>>>>>>> StartApplication call in Slicer3.cxx.  This allows the regular widget
>>>>>>> tear
>>>>>>> down sequence to happen.  The return code is passed back as the exit
>>>>>>> code
>>>>>>> of
>>>>>>> the process.
>>>>>>>
>>>>>>> * if a script is specified using one command line options, it will
>>>>>>> run
>>>>>>> in
>>>>>>> the slicer interpreter (either the tcl or python interpreters,
>>>>>>> depending
>>>>>>> on
>>>>>>> the form of the argument).  The *eval* form of the argument avoids
>>>>>>> building
>>>>>>> the GUI, while the *exec* form builds the GUI and then runs the
>>>>>>> script.
>>>>>>>
>>>>>>> * the --test-mode argument disables any writing to the registry that
>>>>>>> would
>>>>>>> otherwise occur (i.e. the screen size and layout are not saved on
>>>>>>> exit)
>>>>>>>
>>>>>>>
>>>>>>> Happy Holidays,
>>>>>>> -Steve
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>  --evalpython <std::string>
>>>>>>>  Like --exec, but doesn't load slicer first
>>>>>>>
>>>>>>>  --execpython <std::string>
>>>>>>>  Some Python code to execute after slicer loads. (note: cannot
>>>>>>> specify
>>>>>>>  scene after --exec)
>>>>>>>
>>>>>>>  -v <std::string>,  --eval <std::string>
>>>>>>>  Like --exec, but doesn't load slicer first
>>>>>>>
>>>>>>>  -e <std::string>,  --exec <std::string>
>>>>>>>  Some code to execute after slicer loads. (note: cannot specify scene
>>>>>>>  after --exec) (note: use ,. instead of ; between tcl statements)
>>>>>>>
>>>>>>>  -p <std::string>,  --script <std::string>
>>>>>>>  Script to execute after slicer loads
>>>>>>>
>>>>>>>  -f <std::string>,  --file <std::string>
>>>>>>>  Script to execute instead of loading slicer
>>>>>>>
>>>>>>>
>>>>>>>
>
_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
Reply | Threaded
Open this post in threaded view
|

Re: TESTING: Failing Test of the Day (FTOD) (Tcl Guru help needed)

Luis Ibanez
In reply to this post by pieper
Hi Steve,

It turned out that vtkDebugLeaks already has a feature
for forcing the executable to fail (by calling exit(0) ) if
there are leak at the moment of exiting the application.

This is enabled with the call

           vtkDebugLeaks::SetExitError(true);

This line has now been inserted in Applications/GUI/Slicer3.cxx
http://viewvc.slicer.org/viewcvs.cgi/trunk/Applications/GUI/Slicer3.cxx.diff?&r1=11381&r2=11382

With this call, it is now possible to see test failing due
to memory leaks, when running "ctest" locally.


    Luis


----------------------------------------------------------------------------------
On Wed, Dec 23, 2009 at 8:23 PM, Steve Pieper <[hidden email]> wrote:

> Hi Luis -
>
> Regarding singletons: until the pattern was changed very recently -- by a
> Kitware guru ;) -- we were always able to use vtkDebugLeaks to find the
> offending code.  If that is no longer possible then perhaps we should revert
> to the previous singleton pattern which worked well for years and was very
> useful.
>
> Regarding valgrind, I have used it quite a bit with slicer and is useful in
> it's own way (not by itself, but with coffee-enhanced human effort!).
>
> The command line is in a comment in this file:
>
> CMake/Slicer3ValgrindSuppressions.supp
>
> As you mentioned, one needs to use bin/Slicer3-real as the entry point for
> valgrind. It also helps to use the suppression files.  There are many false
> positives from valgrind, which are mostly caught by the supression files
> (but not completely).
>
> -Steve
>
>
>
> Luis Ibanez wrote:
>>
>> Hi Steve,
>>
>>   Yes, let's get them to pass.     :-)
>>
>> After some consultation with Kitware Gurus,
>> it turned out that the use of vtkDebugLeaks
>> is not the proper way of finding the offending
>> code.
>>
>> In particular, the singletons will always be
>> included in the report.   :-/
>>
>>
>> So we are left with using Valgrind (and coffee).
>>
>> My problem with using Valgrind is that Slicer
>> uses the launching mechanism, therefore the
>> executable that we start may not be the one
>> that actually have the leaks.
>>
>> If I use the Slicer3 executable from Slicer-build,
>> Valgrind doesn't find the debugging symbols.
>> I'm forced then to use the executable in
>>
>>           Slicer-build/bin/Slicer3
>>
>> ...but that's not the one we are launching
>> when we run the tests...
>>
>>
>>     Luis
>>
>> -------------------------------------------------------------
>> On Wed, Dec 23, 2009 at 4:36 PM, Steve Pieper <[hidden email]>
>> wrote:
>>>
>>> Hi Luis -
>>>
>>> Thanks for the new tests - now let's get them passing :)
>>>
>>> Yes, we alway use the --test-mode flag on the cmake-based tests.  It
>>> skips
>>> the splash screen and also doesn't save any changes to the registry (and
>>> perhaps a few other things).
>>>
>>> Here's what I get on my build:
>>>
>>>> % ./Slicer3 --no-modules --exec exit
>>>> Setting up launch environment...
>>>> Starting Slicer: /Users/pieper/slicer3/latest/Slicer3-build
>>>> vtkDebugLeaks has detected LEAKS!
>>>> Class "vtkOutputWindow" has 1 instance still around.
>>>> Class "vtkEventBroker" has 1 instance still around.
>>>> Class "vtkFreeTypeUtilities" has 1 instance still around.
>>>> Class "vtkStringArray" has 2 instances still around.
>>>> Class "vtkTimerLog" has 1 instance still around.
>>>>
>>>> There are memory leaks
>>>> child process exited abnormally
>>>>
>>> The vtkFreeTypeUtilities class is odd - perhaps it is created by the
>>> vtkOutputWindow.  The vtkStringArray and vtkTimerLog might also be
>>> associated with this.
>>>
>>> JC recently introduced a new singleton pattern for the vtkEventBroker so
>>> I'm
>>> not sure why that is leaking.
>>>
>>> Typically even a single missing Delete() can lead to a whole cascade of
>>> leaks.  Tracking them down is a bit of a puzzle.  Satisfying when they
>>> get
>>> cleared up, but frustrating that they keep re-appearing.  Perhaps the
>>> vtkSmartPointer conversion isn't such a bad idea...
>>>
>>> -Steve
>>>
>>> Luis Ibanez wrote:
>>>>
>>>> On Wed, Dec 23, 2009 at 2:06 PM, Steve Pieper <[hidden email]>
>>>> wrote:
>>>>>
>>>>> Hi Luis -
>>>>>
>>>>> Yes, it would make a lot of sense to add the following as a test:
>>>>>
>>>>>  ./Slicer3 --no-modules --exec exit
>>>>>
>>>> The test has been added
>>>>
>>>>
>>>> http://viewvc.slicer.org/viewcvs.cgi/trunk/Applications/GUI/Testing/CMakeLists.txt?rev=11378&sortby=date&r2=11378&r1=11255
>>>>
>>>> They are:
>>>>
>>>> * Slicer3CLTest8
>>>> * Slicer3CLTest9
>>>>
>>>> add_test(Slicer3CLTest8 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
>>>> --no-modules)
>>>> add_test(Slicer3CLTest9 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
>>>> --no-modules --exec "exit")
>>>>
>>>> As you pointed out, both are failing
>>>>
>>>> BTW,
>>>> I assume that we also need the "--test-mode" flag,
>>>> is that correct ?
>>>>
>>>>
>>>>> On the current trunk, this test fails :(  No doubt as a result of some
>>>>> recent checkins.  Having this checked automatically will help (assuming
>>>>> people pay attention when the test stops working!!!).
>>>>>
>>>> Could you provide any guess as to how recent the
>>>> offending commit may have been ?
>>>>
>>>> Cleaning up the Dashboard is
>>>> a critical task at this point...
>>>>
>>>>
>>>>> Also, it would be great to add a set of tests, one for each module that
>>>>> is
>>>>> bundled with the slicer build of the form:
>>>>>
>>>>>  ./Slicer3 --ignore-module <ModuleName> --exec exit
>>>>>
>>>>> The other command line arguments would also be great to test.
>>>>>
>>>>>
>>>>> Regarding vtkSmartPointer in slicer, Jim has written some excellent
>>>>> guidelines for how it should be used.
>>>>>
>>>>> http://www.slicer.org/slicerWiki/index.php/Slicer3:Memory_Management
>>>>>
>>>>> The primary development of slicer has been to copy the style of vtk for
>>>>> consistency and the New()/Delete() pattern is widely used inside vtk.
>>>>> Making
>>>>> a change to smart pointers would be a lot of work and probably not the
>>>>> most
>>>>> efficient way to fix the current memory leaks.  But if someone wants to
>>>>> take
>>>>> this on (and does it well) I wouldn't object.
>>>>>
>>>>> Best,
>>>>> Steve
>>>>>
>>>>>
>>>>>
>>>>> Luis Ibanez wrote:
>>>>>>
>>>>>> Hi Steve,
>>>>>>
>>>>>> Thanks a lot for providing more guidelines for testing,
>>>>>>
>>>>>> I was actually wondering a moment ago if the instructions at
>>>>>> http://www.slicer.org/slicerWiki/index.php/Slicer3:Testing
>>>>>> were still valid.
>>>>>>
>>>>>> It seems that some of these suggested tests
>>>>>> are the ones executed  by the tests:
>>>>>>
>>>>>>  ctest -R Slicer3CLTest -N
>>>>>>
>>>>>>  86/ 96 Testing Slicer3CLTest1
>>>>>>  87/ 96 Testing Slicer3CLTest2
>>>>>>  88/ 96 Testing Slicer3CLTest3
>>>>>>  89/ 96 Testing Slicer3CLTest4
>>>>>>  90/ 96 Testing Slicer3CLTest5
>>>>>>  91/ 96 Testing Slicer3CLTest6
>>>>>>  92/ 96 Testing Slicer3CLTest7
>>>>>>
>>>>>> But.. it also seems that we have not all of the
>>>>>> ones suggested in
>>>>>> http://www.slicer.org/slicerWiki/index.php/Slicer3:Testing
>>>>>> and not all the ones in your email.
>>>>>>
>>>>>> Shouldn't we add the missing ones ?
>>>>>>
>>>>>>
>>>>>> The current ones are:
>>>>>>
>>>>>>  ctest -R Slicer3CLTest -N -V
>>>>>>
>>>>>>
>>>>>> 86/ 96 Testing Slicer3CLTest1
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>>> --help
>>>>>>  87/ 96 Testing Slicer3CLTest2
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --xml --test-mode
>>>>>>  88/ 96 Testing Slicer3CLTest3
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode -f
>>>>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/TestScript.tcl
>>>>>>  89/ 96 Testing Slicer3CLTest4
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>>> --eval puts\ testing\ ,.\ exit\ 0
>>>>>>  90/ 96 Testing Slicer3CLTest5
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>>> --exec puts\ testing\ ,.\ exit\ 0
>>>>>>  91/ 96 Testing Slicer3CLTest6
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>>> --script
>>>>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/TestScript.tcl
>>>>>>  92/ 96 Testing Slicer3CLTest7
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>>> --script
>>>>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/LoadSceneTest.tcl
>>>>>>  93/ 96 Testing Slicer3ScrollTest
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>>> --script
>>>>>> /home/ibanez/src/Slicer3/Applications/GUI/Testing/ScrollTesting.tcl
>>>>>>  94/ 96 Testing Slicer3MRMLUndo
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>>> --no-splash -f
>>>>>> /home/ibanez/src/Slicer3-build/share/MRML/Testing/testUndo.tcl
>>>>>>  95/ 96 Testing Slicer3MRMLVolume
>>>>>>
>>>>>> Test command: /home/ibanez/src/Slicer3-build/Slicer3 --test-mode
>>>>>> --no-splash -f
>>>>>> /home/ibanez/src/Slicer3-build/share/MRML/Testing/testVolume.tcl
>>>>>>
>>>>>>
>>>>>> ---
>>>>>>
>>>>>> Also,
>>>>>>
>>>>>> Given the large amount of memory leaks currently
>>>>>> present in Slicer3, I propose to engage in a campaign
>>>>>> of vtkSmartPointer<>  adoption.
>>>>>>
>>>>>> Are there any reasons for not using vtkSmartPointers
>>>>>> massively in Slicer ?
>>>>>>
>>>>>>
>>>>>>   Thanks,
>>>>>>
>>>>>>
>>>>>>        Luis
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------------
>>>>>> On Tue, Dec 22, 2009 at 7:42 PM, Steve Pieper <[hidden email]>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi Luis -
>>>>>>>
>>>>>>> Many thanks for stepping up as the champion of testing.  I would also
>>>>>>> like
>>>>>>> to encourage everyone to take creating and fixing tests seriously.
>>>>>>>  It
>>>>>>> will
>>>>>>> make all of our lives easier!
>>>>>>>
>>>>>>> Regarding this simplest of all tests that you mentioned:
>>>>>>>
>>>>>>>> Could you please suggest what should be
>>>>>>>> added to the script in order to make a clean
>>>>>>>> return  ?
>>>>>>>>
>>>>>>>> We could replace the current code in:
>>>>>>>>
>>>>>>>> Applications/GUI/Testing/CMakeLists.txt:
>>>>>>>>
>>>>>>>> add_test(Slicer3CLTest5 ${Slicer3_BINARY_DIR}/Slicer3 --test-mode
>>>>>>>> --exec "puts testing ,. exit 0")
>>>>>>>>
>>>>>>> This test simply lets slicer start up then exits cleanly.  This test
>>>>>>> is
>>>>>>> a
>>>>>>> good way to see if the GUI startup up and shuts down without crashing
>>>>>>> or
>>>>>>> leaking.  I don't think the test needs to be changed.  In fact, I'll
>>>>>>> bet
>>>>>>> it
>>>>>>> touches a lot of code that has to all work right for this to pass
>>>>>>> (module
>>>>>>> discovery, GUI startup, scene creation and deletion...).
>>>>>>>
>>>>>>> The tests are all passing in the Slicer-3-4 branch (including this
>>>>>>> simple
>>>>>>> one).  Things have gotten broken in the current trunk and need to be
>>>>>>> fixed.
>>>>>>>
>>>>>>> -----
>>>>>>>
>>>>>>> Slicer developers may want to consider running the following handy
>>>>>>> command
>>>>>>> lines before checking in code:
>>>>>>>
>>>>>>>  ./Slicer3 --exec exit
>>>>>>>
>>>>>>> (this checks that all of slicer will start and exit cleanly)
>>>>>>>
>>>>>>>  ./Slicer3 --no-modules --exec exit
>>>>>>>
>>>>>>> (this will test just the base of slicer: no modules will be loaded or
>>>>>>> built).
>>>>>>>
>>>>>>>  ./Slicer3 --ignore-module <ModuleName> --exec exit
>>>>>>>
>>>>>>> (this will specifically leave out a module to help identify where a
>>>>>>> leak
>>>>>>>  or
>>>>>>> crash is coming from.  Note that the module's GUI class constructor
>>>>>>> will
>>>>>>> be
>>>>>>> called, but not the BuildGUI method, so can still generate test
>>>>>>> failures
>>>>>>> even if you "--ignore" it).
>>>>>>>
>>>>>>> With these you should be able to test
>>>>>>>
>>>>>>> -----
>>>>>>>
>>>>>>> Some notes on the testing script options for slicer (see the variants
>>>>>>> from
>>>>>>> --help below - these are used in many of the tests currently included
>>>>>>> in
>>>>>>> slicer):
>>>>>>>
>>>>>>> * the standard tcl 'exit [code]' keyword is overridden.  Instead of
>>>>>>> exiting
>>>>>>> right away, slicer's exit will return the flow of control to after
>>>>>>> the
>>>>>>> StartApplication call in Slicer3.cxx.  This allows the regular widget
>>>>>>> tear
>>>>>>> down sequence to happen.  The return code is passed back as the exit
>>>>>>> code
>>>>>>> of
>>>>>>> the process.
>>>>>>>
>>>>>>> * if a script is specified using one command line options, it will
>>>>>>> run
>>>>>>> in
>>>>>>> the slicer interpreter (either the tcl or python interpreters,
>>>>>>> depending
>>>>>>> on
>>>>>>> the form of the argument).  The *eval* form of the argument avoids
>>>>>>> building
>>>>>>> the GUI, while the *exec* form builds the GUI and then runs the
>>>>>>> script.
>>>>>>>
>>>>>>> * the --test-mode argument disables any writing to the registry that
>>>>>>> would
>>>>>>> otherwise occur (i.e. the screen size and layout are not saved on
>>>>>>> exit)
>>>>>>>
>>>>>>>
>>>>>>> Happy Holidays,
>>>>>>> -Steve
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>  --evalpython <std::string>
>>>>>>>  Like --exec, but doesn't load slicer first
>>>>>>>
>>>>>>>  --execpython <std::string>
>>>>>>>  Some Python code to execute after slicer loads. (note: cannot
>>>>>>> specify
>>>>>>>  scene after --exec)
>>>>>>>
>>>>>>>  -v <std::string>,  --eval <std::string>
>>>>>>>  Like --exec, but doesn't load slicer first
>>>>>>>
>>>>>>>  -e <std::string>,  --exec <std::string>
>>>>>>>  Some code to execute after slicer loads. (note: cannot specify scene
>>>>>>>  after --exec) (note: use ,. instead of ; between tcl statements)
>>>>>>>
>>>>>>>  -p <std::string>,  --script <std::string>
>>>>>>>  Script to execute after slicer loads
>>>>>>>
>>>>>>>  -f <std::string>,  --file <std::string>
>>>>>>>  Script to execute instead of loading slicer
>>>>>>>
>>>>>>>
>>>>>>>
>
_______________________________________________
slicer-devel mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
To unsubscribe: send email to [hidden email] with unsubscribe as the subject