Running unit tests and Syncing (or not) with CVS...

The tests finally run!

Dirk came through with the magic answer in an e-mail right away after my last request.'

- - - - - - -

John -you need to have org.eclipse.test.performance in your workspace. You can find the project in the CVS repository.
I tested the following setup:
- fresh workspace
- checked out org.eclipse.test.performance and org.eclipse.jdt.ui.tests
- executed org.eclipse.jdt.ui.tests.wizardapi.NewTypeWizardTest
Dirk

- - - - - - - - - -

So I tried it, and by golly it worked like a charm. I experimented with running the tests some, and e-mailed Dirk back:

- - - - - - - - - -

Dirk - Hey it's working! Thanks for the tip.

BTW - I got the first little part of the change working - a new ${type_content} code template that appears on the Window->Preferences.. dialog. Next I need to do some refactoring of the NewTypeWizardPage internals to get the ${type_declaration_header} variable to work. But now I can do regression tests, so this really unblocks me.

I may have a question about CVS syncing if I can't figure it out on my own. Otherwise, I'll keep you informed of progress as I go.

- John

- - - - - - - - - - - -

There is a suite of tests in the org.eclipse.jdt.ui.tests package called "AutomatedSuite.java". This suite has 952 junit tests in it, and takes a little under 15 minutes to run on my Dell laptop. The tests all run green (no errors) with the latest Eclipse integ build.

If you try to run every test in the larger "ui" package within the jdt tests directory, there are 3170, it takes about 3 hours, and there are around 80 errors and failures. I wrote 2 e-mails to Dirk to clear up expectations of how I am supposed to work with these things.

Dirk, I'm on to my next round of questions. I have figured out how to sync back with CVS, so I don't think that will be a problem, but I've run the test suite a few times in different combinations, and I want to make sure I'm running it right.

There's a test class in the org.eclipse.jdt.ui.tests package called "AutomatedSuite" that has 952 junit tests in it. It is running green (all tests pass) in the latest integ build (20050906-800). I just wanted to confirm this is the right test suite to run to check for regressions. I noticed there are other tests that are included, but are not in this suite, and they do not run green. If I run everything (just select the "ui" package and run it all as a plug in test) there are over 3100 tests, and around 80 failures and errors.

I just wanted to double-check it was the expectation that people should run the "AutomatedSuite" tests and that these tests are expected to run with no errors for each integ. If that is the case, then development for folks like me is much simpler. If there's another suite I'm supposed to run, let me know. Running everything and keeping track of which failures were there and which failures I caused is quite painful, and I just want to make sure that's not your expectation.
Thanks,
- John

- - - - - - - - - -

John,

running AutomatedSuite is the correct thing to do. It is the root suite forall ui related JDt tests. There is another one for refactoring tests. It iscalled AllAllTests. However for your work on the New wizard it is enoughthe run the AutomatedSuite regulary. Before providing the patch you can runthe AllAllTests once to do a last check. However the refactoring tests are> 3000 and take some time ;-).

Dirk

- - - - - - - - - - -

Oh - one more question - does the passing of junit tests have anything to do with whether the integ is marked with a green check mark or a red "x" in the download site? I noticed that sometimes different platform builds have either checks or x's. What's the code for that?

Thanks,
- John

- - - - - - - - - -

Yes, if the build has a red x at least one test has failed. You can clickon the red X which will take you to a list of test results you can furtherinspect to see what test has failed. Currently we are running the tests onethree different platform and it may be they fail only on one.
Dirk

- - - - - - - - - -

CVS and not Syncing

Well, I was wrong about syncing up with CVS - I couldn't find the formula to do it after all. There are commands in Eclipse to sync to head, but since the head revisions don't build (at least I don't think they do...) I didn't try that. I've been working with an integ build, and wanted to update my local build to the next build while retaining my changes. That's a pretty normal thing to do with CVS, but I couldn't find the formula in the Eclipse integration. In CVS, you do an "Update" command, and choose the label you want to update to. In Eclipse, the "Update" menu command (and the "Synchronize with repository" for that matter) don't allow you to specify the label you want to sync to (or update from).

So I went to WinCVS on my machine, and tried to update from there. The "Update" command was there as I expected, with options. I tried it, and it seemed to work. It gave it's usual reams of output where it copied files from the server to my box, and it seemed to do the merges right for the files I had changed. That's when I wrote the e-mail above that syncing would be "no problem."

But then, back in Eclipse, I tried to rebuild, and it thrashed around in pitiful death throes for a few seconds before puking out a gazillion errors and saying "rosebud...." So, with no other bright ideas to fall back on, I got a new integ, hand-checked the diffs to my files, and copied them over. (I was at my daugther's soccer practice with no good diff-merge utility on my laptop, so not only did I have to do this tedious work with MS-Word, I looked like a real nerd to the other parents sitting in the shade tapping on my Dell D-800 brick and muttering to myself about sub-standard tools.) Hope I get this fixed (the syncing, that is - there's no hope for me to stop looking like a nerd. Hell, I love being a nerd - that's why I do this.)

Got it all going and played around with some more coding and prototyping.

My addition of the ${type_content} variable broke two of the 952 jdt junit tests, both in TemplateStoreTest.java. Adding a new code template variable changes the count of variables, and you have to inform the tests of your new variable by including it in a "ALL_CODE_TEMPLATES" member array that's defined near the top of the class. (But not at the top of the class! Somebody's got to review the Sun Java coding standards....)

I tried to work a new variable through the calling sequence of the existing classes, and that was enough of a prototype for me to smell trouble. Next up, I have to write to the Eclipse committers to convince them the current class factoring is too hosed up to make this change, and needs refactoring and deprecating the current API.

That done, I reverted my changes to the "NewClassWizard" and "StubUtility" classes. It's tricky to find how to revert if you want to cancel changes you've made. In the CVS Repository perspective, you can't browse and select the file you want in the view on the left and find a menu item to revert it. But you can search for the file using the "Search->file" menu, then right click on it in the bottom "search" results view. If you right-click the file in the window, the pop-up menu has a "Replace with..." sub-menu, and one of the menu items allows you to replace the file with the version you last got from CVS.