And the winner is.....

>> Monday, December 14, 2009

Eric Rizzo has won the December splash screen contest with this submission.


Congratulations! Thanks to all the contestants for submitting such creative artwork. The winning design will start appearing in tonight's build (N20091214-2000) and has been tagged for the integration build.

Read more...

Vote now for your December Splash

>> Friday, December 11, 2009

Next week, we'll have the our final integration build before the end of 2009. What better way to end the year than with a custom splash screen? As usual, the very creative Eclipse community has stood up to the challenge with eight different designs for your consideration. Please vote here:



Direct link to poll.

The poll closes on Monday at 5 pm EST. I'll announce the winner on Monday and commit the winning splash. Good luck to all the contest entrants!

This contest is closing a bit earlier than expected because we shutdown our builds over the holiday season. Thus, we need to get the December splash into the integration build on December 15, 2009.

Read more...

Eclipse and Equinox 3.6M4 now available




Ho ho ho.
Ho ho ho.
We are Eclipse's elves.

We are Eclipse's elves,
Filling mirrors with M4,
Another milestone out the door.

Oh, we are Eclipse's elves,
We work hard all day,
Debug and refactor is our play,
Bugs we stamp out,
Hurray, the community shouts!

We are Eclipse's elves,
We ship on time each year,
We don't like to brag,
Just sync, commit and tag,
Install new bundles without lag.

We all know who's been good,
Resolved the bugs you should,
Eclipse is you,
Grab a bug and become a committer too!

We are Eclipse's elves.
Ho ho ho. Ho ho ho.
We fix the code ourselves.
Ho Ho!

The Eclipse and Equinox team are happy to announce the release of the Eclipse 3.6M4 milestone.

New and noteworthy

Update to 3.6M4 by adding this site to your list of available sites
http://download.eclipse.org/eclipse/updates/3.6milestones

Equinox Downloads

Eclipse Downloads

Happy Holidays and New Year from the Eclipse and Equinox projects!

Read more...

December splash screen contest: Prizes++

>> Wednesday, December 09, 2009

Last week, I announced the December splash screen contest. Since then, we've had some entries from Olivier Thomann. Very creative. Look at this


Or this.


We need more people to submit splash screens to to the contest. Here's the bug where you can submit your entry. It doesn't have to be a holiday theme, it can be whatever you'd like.

As an added bonus, the generous folks at the Eclipse foundation (hi Ian and Lynn) have offered to donate some eclipse bling to the prize pot. So the total prize package for the December splash screen contest will be

1. Friends of Eclipse membership
2. Eclipse t-shirt
3. $50 donation to Eclipse in your name

An Eclipse t-shirt? Does it give you special powers like the famous "three wolves t-shirt"?




I guess you'll have to enter the contest to find out. Fire up your favourite graphics editor and submit a splash screen to bug 296918

Read more...

A tale of CVS, NFS and Swordplay

>> Friday, December 04, 2009

Earlier this year, the time that it took our builds to check out code from CVS had slowed to a crawl. Last year, our integration builds that started by 8am were available by noon. However, during the 3.6 cycle, most builds weren't ready until 2pm or even 4pm. Very painful, especially for our European committer friends. Not only were the builds slow, but it took a long time for us to commit code, and timeouts were frequent.

Denis and his merry band of webmasters determined that the source of the problem was twofold.

1) NFS configuration issues causing high CPU load on the servers. (Bug 288293)
2) Anonymous pserver users were holding the cvs lock files for a long time (Bug 293355)

Earlier this week, Denis fixed the NFS configuration issues. This had reduced the time it takes to check out our code significantly. Also, I haven't seen any CVS timeouts lately. Today, our 8am integration build was ready at noon. A big thank you to the webmasters! We are much more productive thanks to your work. Less swordplay while waiting for builds, and more time to fix bugs and implement new features. Again from xkcd.



Really, who doesn't love xkcd? It's so good.

Denis will change the anonymous pserver access to a mirrored copy of the CVS repository on December 11. He sent notes to the committers list with the details. The means that committers won't have to contend with anonymous users for cvs lock files if they check out code via ssh. Psever will be available on a separate file system, but sychronized with the live copy.

In preparation for this change, I modified our build scripts to check out our code using ssh, instead of anonymous pserver. However, our map files remain untouched so anyone can check out our code via pserver. The build scripts modify the map files after they have been checked out. We won't have to deal with file contention for CVS lock files. And ssh connections to eclipse.org have a higher QoS than pserver. Good stuff.

Most teams won't have to make any changes because they run their builds on build.eclipse.org. These users will still pull from the live copy of the data via pserver. And the mirrored copy will be up to date. I'm just trying to do everything I can to avoid infrastructure issues that cause build failures.

See bug 294900 for the platform releng changes.

Read more...

Make a splash in December

The Movember splash was a lot of fun. I'd like to open up submissions for a December splash screen. It will be released after M4. This contest is open from December 4-15. At that time, I'll start a poll and ask the community to vote on the splash screen. I'll commit the winning submission. What will you win? Eyeballs on your art. Respect of the community. Warm and fuzzy feelings. And I'll donate $50 to friends of Eclipse in your name.



The existing splash screen is in org.eclipse.platform in /cvsroot/eclipse. The file is splash.bmp. The photoshop files are in 3_6SplashHeliosPsd.zip in the same project. Attach your entries to bug 296918

Nothing offensive please people. Go for it.

Read more...

Good Eclipse Swag

>> Monday, November 30, 2009

I found this mug in the kitchen at work today. It has Eclipse key bindings on front


and back.



Now, all I need is a sandwich to go with this and I'd be set. From xkcd




Swag is rarely both useful and aesthetically appealing. This is both! Thanks itemis!

Read more...

The end of a three year relationship: Goodbye bug 153429

>> Friday, November 27, 2009

This past week, the Eclipse and Equinox team released changes to support the use the JUnit 4 bundle in the Eclipse Test framework in bug 153429. This bug had over 75 people on the cc list and 40 votes to fix it. It was in my bucket for three years. It's finally fixed so now we and (you) can run your automated tests on JUnit 4.

The changes that we made to fix this bug are described in the wiki. John also sent a message to the cross project list.

I have worked on many bugs during my tenure at Eclipse. This was a tough one. It was very complex, and required a tremendous amount of testing. It was also very much a team effort.

I'd like to especially thank DJ Houghton for all his patches for the test harness and and test bundles. Also, DJ added the org.junit 4.7.0 bundle to Orbit which was very helpful. John Arthorne for fixing the p2 tests and all the good advice during this process. Dani Megert and Markus Keller for their changes to JDT. Darin Wright for his advice regarding PDE. Curtis Windatt for his changes to the pde ui tests. Andrew Niefer for the changes to the pde build tests. Other committers made minor changes to their test bundles to accommodate this change. Thank you!

Goodbye bug 153429. I won't miss you.

Read more...

Committer Reps, we need your help

>> Wednesday, November 25, 2009

Hi Boris, Chris, Doug and Ed

Hey, how's it going. I hope that you are all well.

Like a mall on Black Friday, around here it's peak Eclipse committing season. Lots of bugs to fix. Lots of builds to run. As you know, builds are quite a pain point at Eclipse. I'm excited about the possibilities in the b3 project to make things better. Building software is complex, and Eclipse is no exception. In additional to tooling to make builds easier, we need hardware to make builds faster. Our build today takes about five hours to complete, and an additional 6.5 hours for the tests to complete. Really, it's not pretty.

Many eclipse projects run their builds on Hudson on build.eclipse.org. Hudson is fantastic because there's a rich set of plugins that you can use to enhance the functionality of your build. Also, since this server has local access to the eclipse.org filesystem for code checkouts, you're less prone to network errors which can break the build. It also has ldap integration with your commiter login so you can restrict your build configuration to the commiters on your project. In theory, if you need more build machines to run your build - you can use the Amazon EC2 plugin to provision more machines in the cloud, or other plugins to start builds on local slave machines. Good stuff.

However, one of the things that the foundation doesn't provide today is test machines. This means that we can't run our build at the Eclipse foundation. The Eclipse Project builds zips for 14 different platforms. We run JUnit tests on three native platforms: Windows, Linux and Mac. They are the most commonly downloaded platforms. We need test machines to ensure that we don't have any bugs specific to a platform. Why do our tests take so long? We have 54,000 JUnit tests. You don't produce quality software by skimping on tests.

This isn't just about the Eclipse and Equinox projects. This could be very useful for other projects, for instance, the XSL tools project has expressed interest in using test servers. In addition, these machines could be used as slaves machines for running the build in the event that the main Hudson server is too busy. If we had enough machines, we could run more tests in parallel and reduce the time it takes our build to complete. This would be a big win for the community and our committers.

One thing I investigated in running tests in the cloud. However, most cloud services don't have provide a way to run tests on Macs and we need to make sure that our Mac users are happy. If there is a way, I'd appreciate a link. In addition, one of the advantages of running tests on machines local to the eclipse.org filesystem is that we don't spend time copying stuff back and forth across the network. It's just there.

So, what I'm asking from you is at the next board meeting, please bring up the issue of funding test infrastructure at the Eclipse foundation. It might be even be an advertising opportunity for one of the member companies if they donated hardware. Other companies could donate money to pay for the additional rack space. I don't know right now what the final technical solution will be or what it will cost. All I'm asking right now is to start the conversation.

For many years, the Eclipse project has been criticized for not being open enough. Having our build process fully on eclipse.org servers would make us more open. It would also allow any of the Eclipse and Equinox project committers, regardless of company affiliation, to initiate a build. It we had enough hardware, our build could be faster and we could spend less time waiting for builds, and more time fixing bugs the builds reveal.

Please bring this issue up and the next board meeting.

sincerely,
Kim

P.S. Right now we have the following test machines and our tests take about 6-8 hours to complete. Obviously, if we had more machines running tests in parallel, the build would take less time.
1) JUnit: 2 linux, 2 windows, 1 mac, 1 test cvs server,
2) Performance: 2 windows, 2 linux, 1 database server

Read more...

Happy Birthday Eclipse...now some pictures from your youth

>> Tuesday, November 03, 2009

Saturday November 7th, 2009 is Eclipse's 8th birthday. Eight years ago the first Eclipse downloads were publicly available along with the source code.

Here's a picture of the first www.eclipse.org. The website was slashdotted shortly after the "Eclipse is open source" announcement was made. The hard drive had to be replaced, thus the missing cover. This machine was replaced by a real server a few weeks later. A couple of years later, this was replaced by faster and more fault tolerant hardware managed by our most excellent webmasters.



I went through my desk the other day, and found some emails of the original requirements for the eclipse infrastructure, project structure and commit rights. A fast server has 512MB RAM and 20GB of disk space...really? Okay, I feel old.



Bugzilla was there from the start.



Some of the first mailing lists. A few disappeared but even more new ones were created. Today, we have eclipse on twitter, forums, conferences, marketplace, blogs and the list goes on. Pretty impressive!



In 2001, the linux kernel still fit on a floppy disk. It was useful to have a boot disk in the event that the boot partition became corrupted and the machine wouldn't boot. Much easier than booting from a rescue cd and mounting all the partitions by hand. Especially at 4am. Here's a picture for those who've never worked with a diskette (*ahem* co-ops :).




Initial project structure. Today we have so much much more.


The first commit rights - hello PDE family! Today we have an amazing diversity of committers from around the world and many companies. Eclipse also has friends, but would always like more....who doesn't?




In reality, the past doesn't matter that much, and we need to focus on the future. Sometimes I'm cynical about Eclipse, because as a long time committer, I notice that many people like to play (consume), but fewer want to pay (contribute). And to some degree we make it difficult to contribute without a significant investment of time and a steep learning curve. But most days, I'm absolutely amazed by the work that we as a community can do together, when smart, passionate people strive toward a common goal. Happy Birthday Eclipse!


Read more...

Now I see IU, now IU don't

Eclipse 3.6M3 went out the door over the weekend, along with a lot of Halloween candy.



Sometimes, you can have too much Halloween candy. And sometimes, you can have too many IUs in your p2 repo. Don't believe me - just look at this repo with bogus bundles - scary!





Both scenarios can cause your friendly neighbourhood release engineer pain. This is unusual because we're a very pain tolerant people. To alleviate the suffering, the p2 team added an Ant task in 3.6M3 that allows you to remove bundles from your repo. As much as I love spending quality time at the command line modifying metadata, Ant tasks that automate tedious jobs are even better.

The p2.remove.iu task will remove both the metadata and the bundle from the repository for a specified IU. For example, if you had bogus packed com.ibm.icu.* bundles in your repo, this task would remove them.

<p2.remove.iu>
<repository location="file://${reposource}" />
<iu id="com.ibm.icu" artifacts="(format=packed)" />
<iu id="com.ibm.icu.base" artifacts="(format=packed)" />
<iu id="com.ibm.icu.source" artifacts="(format=packed)" />
<iu id="com.ibm.icu.base.source" artifacts="(format=packed)" />
</p2.remove.iu>

This task is useful if you'd like to remove some built time bundles from your repo. Or just correct a mistake after a release. It happens. In any case, it's all good. Almost as tasty as chocolate.




Related bugs

Support excluding bundles when running p2.process.artifacts task
p2.remove.iu task should have an option to specify to remove packed file only
ICU jars at Eclipse 3.5 update site have size of 0

Read more...

Hudson is sweet, now the build can tweet

>> Wednesday, October 21, 2009

I recently set up a new Hudson build so the Equinox team could test their changes in a branch instead of releasing everything to HEAD. Hudson, like Eclipse, has a rich variety of plugins that can expand the functionality of your build. Hudson has a twitter plugin and thus Eclipse (test) builds tweet here

http://twitter.com/eclipsebuilds

Right now, there are only a few messages about the test builds I started, and stopped when I saw that build was proceeding as normal. In any case, it's pretty cool.

Hudson also has plugins to use additional slave machines on the network to run the build on a faster machine if the current build machine is too busy. It also has plugins to provision Amazon EC2 images and run the build on a slave in the cloud. We currently run about 54,000 JUnit test per platform multipled by five test machines during each build. We run tests on Windows, Mac and Linux machines. But the tests take a long time to complete simply because of the number of tests we run and the lack of additional hardware to run more tests in parallel. It would be fantastic to be able to run the tests on parallel on many machines and finish in fraction of the time they take today. That would allow us to reduce the number of breaking build issues. But of course, this will take significant testing to implement. Sounds like fun!

References
Run JUnit tests in parallel
Hudson build for the p2 branch

Read more...

Test bundles switch from runnable to repo

>> Tuesday, October 20, 2009

Earlier this week, I changed the build so that the test bundles are provided in a zipped p2 repository format. A p2 repository looks something like this

...
artifacts.jar
content.jar
features/org.eclipse.sdk.tests_3.6.0.N20091019-1735-9J9fG6sFIKSg8a7j2ZwWY0UAV4BV.jar
plugins/org.eclipse.ua.tests_3.3.300.N20091019-1735.jar
plugins/org.eclipse.jdt.debug.tests_3.1.100.N20091019-1735.jar
binary/
...

The build will install the appropriate test bundles in the eclipse install being tested using the p2 director. Previously, our test bundles were assembled in the runnable format like this

...
eclipse/plugins/org.eclipse.jdt.debug.tests_3.1.100.N20091019-1735/...
eclipse/plugins/org.eclipse.ua.tests_3.3.300.N20091019-1735/..
eclipse/features/org.eclipse.sdk.tests_3.6.0.N20091019-1735-9J9fG6sFIKSg8a7j2ZwWY0UAV4BV
...

and unzipped into the dropins folder. However, the dropins folder is really for legacy purposes. Disadvantages of using the dropins folder include

  • It's to help those applications that expect that dropping a bunch of bundles into an install will work (!).
  • All the bundles in the dropins folder are treated as optional bundles.
  • You can't update the bundles in the dropins folder by using the UI, you are responsible for provisioning them.

The content of the org.eclipse.test framework bundle hasn't changed. However, if you're rerunning the Eclipse and Equinox projects' tests JUnit tests against your product, you'll need to install them into the eclipse your are testing using the p2 director. Unzipping the our JUnit bundles into the dropins folder won't work. Alternatively, you could use the reporunnable task to transform the repository into the old runnable format.

Why make this change? These days, I would expect that most users are installing bundles via a repository such as Galileo or one offered by a product team. Unzipping files is so Eclipse 1.0. So this will allow our tests to replicate the environment that reflects the reality of our users. As well, as I mentioned in my earlier post, we really need to run our tests in parallel on more machines to speed up the build process. Having the test bundles available in a repository, eventually in a shared location, is a step toward that goal.


References

Test bundles should be packaged as a repo


Supported dropins formats

repo2runnable task

Note: The Eclipse and Equinox projects provide their JUnit test bundles in the eclipse-Automated-Tests-${buildId}.zip in the eclipse-testing/eclipse-junit-tests-${buildId}.zip file that is available with every build.

Read more...

Galileo SR1 has left the station

>> Friday, September 25, 2009




The Eclipse and Equinox teams are pleased to announce that our 3.5.1 release now available as part of the Galielo SR1 release. Congratulations to all contributors and committers whose hard work made this release possible!

Galileo

3.5.1 Readme

Eclipse

Equinox

You can also update to 3.5.1 using Help->Check for Updates

Enjoy!

Read more...

Eclipse Helios: More repos, fewer zips

>> Thursday, July 23, 2009

In the 3.5 cycle, we introduced zipped p2 repositories in the build. These zips could be used to provision your target platform or update your install. We left the per-platform zips on the download page for Galileo. With 15 platforms multiplied by several permutations of features, and runtime versus source downloads, these zips consumed significant space on eclipse.org and the mirrors. Not to mention the fact that platforms other than Windows, Linux and Mac have very low download numbers.

For Helios, I've replaced all the platform specific RCP, JDT, PDE, CVS, examples and releng tools zips with p2 repositories specific to that feature. For instance, instead of 15 platform specific runtime RCP zips, there's a p2 source and runtime feature for each one. The SDK and Platform feature zips for each os.ws.arch combination continue to be built.

More information
How to Consume zipped repositories.
bug 280622

Read more...

Galileo Release: Eclipse and Equinox 3.5 now available

>> Monday, June 22, 2009

After a year of development, Eclipse and Equinox 3.5 are now available for download. Please put your feet up and enjoy our new release.


Galileo, Eclipse, and Equinox pages.


Every release is different and this year is no exception.

There's planning and committing to a goal.



Learning new things. See the New and Noteworthy and the readme.

Discussions with committers and community. How many rebuilds will there be?




Drama and angst.



Making new friends. We'd especially like to thank these people who really contributed to the success of this release.



Other Eclipse friends decide to see what's behind a new door.




Some days are a blur.



The final steps at the end of the release seem to take forever.



But the end is sweet.




Congratulations to the Galileo teams on the release. I hope everyone takes time to relax. Because it won't be long before have to lace up your shoes and get ready to run towards Helios!

Read more...

Adventures in Publishing

>> Friday, June 19, 2009

For Eclipse 3.5M7, we changed the build to take advantage of the new p2 publisher to to simplify the creation of the platform, RCP and SDK zips. Why switch to the publisher?

  • Metadata is generated on the source when pde build is invoked, instead of after the jars are built.

  • The publisher provides better support for the p2.inf

  • The metadata generator tasks are being deprecated

  • The publisher makes adding a new platform to the build much easier because metadata is generated for the root files. (We added five new platforms in the 3.5 cycle).


  • Also, the Equinox team also wanted to test the publisher in an active build environment and iron out any issues this might reveal. Similar to how Equinox is the reference implementation for OSGi, platform releng is the "reference build" for the Equinox team ;-).

    Previous to using the publisher, we built the sdk feature and the associated root file feature to construct the zips available on our build page. Root files are just files that reside at the root of the eclipse install such as the eclipse executables, config.ini, eclipse.ini and license files.

    To switch the our build to using the publisher I implemented the following steps

    1) Set p2.gathering=true in the build.properties of the builder for our master feature. The master feature contains all the bundles and features we compile in the build. I also set skipMirroring=true, since we'd like to continue to control the mirroring aspects later on in the build.

    2) Remove the call the p2.generate task after your bundles are built. While using the publisher, the metadata is generated when pde build is invoked in your headless build. So you don't need the separate generate call anymore.

    3) Switch the SDK a feature based to product based build. This isn't necessary for most projects that are switching to the publisher. However, the Eclipse SDK is a collection of artifacts for many platforms and product builds make life easier. We also had to to write a p2.inf file to include some platform specific bits into the product build since the PDE UI's product editor has support for four platforms. We build 16! There's a bug open to support more platforms in product files, hopefully this will be addressed in 3.6.

    4) Run the appropriate p2 publisher task. In our case, since we have product builds, I run the p2.publish.product task to publish metadata for the products to the repository.

    <p2.publish.product flavor="tooling" repository="file:${reposource}" productFile="${eclipse.build.configs}/sdk/builder/sdk.product" compress="true">
    <config os="win32" ws="win32" arch="x86" />
    <config os="win32" ws="win32" arch="x86_64" />
    <config os="win32" ws="wpf" arch="x86" />
    <config os="linux" ws="gtk" arch="x86" />
    <config os="linux" ws="gtk" arch="x86_64" />
    <config os="linux" ws="gtk" arch="ppc" />
    <config os="linux" ws="gtk" arch="s390" />
    <config os="linux" ws="gtk" arch="s390x" />
    <config os="linux" ws="motif" arch="x86" />
    <config os="solaris" ws="gtk" arch="sparc" />
    <config os="solaris" ws="gtk" arch="x86" />
    <config os="macosx" ws="cocoa" arch="x86" />
    <config os="macosx" ws="cocoa" arch="x86_64" />
    <config os="macosx" ws="carbon" arch="x86" />
    <config os="hpux" ws="motif" arch="ia64_32" />
    <config os="aix" ws="motif" arch="ppc" />
    <advice kind="featureVersions" file="${buildDirectory}/finalFeaturesVersions.properties" />
    <advice kind="pluginVersions" file="${buildDirectory}/finalPluginsVersions.properties" />
    </p2.publish.product>


    5) Update any custom build scripts with the publisher targets. The pde build included in 3.5 will generate this automatically for you. However, if you have bundles where build.properties indicates custom=true, you'll have to update your static build.xml files to include the new publishing parts. We use custom build scripts for our doc and swt bundles. See this patch on bug 266488 for an example. Basically, you're adding the following to the init target of your build.xml.

    <condition property="p2.publish.parts" value="true">
    <istrue value="${p2.gathering}">
    </istrue>
    </condition>

    and then this additional target.

    <target name="publish.bin.parts" depends="init" if="p2.publish.parts">
    <antcall target="gather.bin.parts">
    </antcall></target></pre><param name="destination.temp.folder" value="${build.result.folder}">

    <eclipse.gatherbundle metadatarepository="${p2.build.repo}" artifactrepository="${p2.build.repo}" buildresultfolder="${build.result.folder}" targetfolder="${build.result.folder}/${full.name}">
    </eclipse.gatherbundle>
    </target>

    After these steps are completed, the build proceeds as normal to provision the zips from the repository using the p2 director.

    Further References

    If you're contemplating switching to the publisher, these documents are useful references. Our build is quite complicated with it's 16 platforms and associated launchers, the transition for most teams will be much simpler than our scenario.

    The meta bug for the changes to reorganize the Eclipse and Equinox build to use the publisher.

    Wiki article on the p2 publisher

    Wiki article on customizing metadata

    Ian's blog entries about the publisher 1 and 2.

    Read more...

    Categories with Composite Repos

    >> Thursday, May 21, 2009

    Categories are groupings of elements that are available for installation in the p2 user ui. For example, the categories in the Eclipse and Equinox site look something like this...



    Ensuring that categories are generated in the content.jar has always been a bit tricky. You can generate a buildtime site.xml that's used to specify the categories which is passed as an argument to the generator. However, this site.xml isn't actually used in the repo because that's old update manager technology. So 2007. You can also add the category name in a p2.inf. Again, not the most intuitive approach.

    Today, John Arthorne proposed an interesting solution to this issue in bug 277359 that doesn't require changes to the build scripts. The trick is to add a child metadata repository to your composite repository that only specifies your category IUs. The steps are as follows:

    Add a "categories" directory to your composite repository

    compositeArtifacts.jar
    compositeContent.jar
    I20090518-2000/
    I20090519-2000/
    I20090520-2000/
    categories/
    In the categories directory, add a content.jar that specifies the IUs of the of the categories you want to display. For the Eclipse project's SDK category, the content.xml looks something like this...

    <unit id="org.eclipse.sdk.ide.categoryIU" version="0.0.0">
    <properties size="2">
    <property name="org.eclipse.equinox.p2.name" value="Eclipse SDK">
    <property name="org.eclipse.equinox.p2.type.category" value="true">
    </property>
    <provides size="1">
    <provided namespace="org.eclipse.equinox.p2.iu" name="org.eclipse.sdk.ide.categoryIU" version="0.0.0">
    </provided>
    <requires size="1">
    <required namespace="org.eclipse.equinox.p2.iu" name="org.eclipse.sdk.ide" range="0.0.0">
    </required>
    <touchpoint id="null" version="0.0.0">
    </touchpoint>

    The final task is to add the child repo "categories" to your compositeContent.jar in the root of your composite repository. You can use the p2 ant tasks to add the child repo or modify the CompositeContent.xml directly.

    <children size="4">
    <child location="I20090518-2000">
    <child location="I20090519-2000">
    <child location="I20090520-2000">
    <child location="categories">
    </children>

    For more information, see bug 277359, the Eclipse and Equinox projects' integration build repo's categories content.jar and compositeContent.jar.

    Read more...

    Ada Lovelace Day: One day late

    >> Wednesday, March 25, 2009

    As Gorkem mentioned, yesterday was Ada Lovelace day. Thank you Gorkem for mentioning me among women in the Eclipse community. There are a few more I'd like to thank for their excellent work.

    • Kim Horne is uber talented Eclipse UI developer who recently moved on to work at RIM.
    • Sonia Dimitrov is an amazing release engineer who now works on Jazz.net by day, while playing violin by night.
    • Carolyn MacLeod is a very accomplished SWT committer who is an authority on accessibility.
    • Susan McCourt is responsible for all the work with the P2 user ui, and in the past provided her many talents to the Platform UI team. The p2 user ui is substantially changed in 3.5 as described in today's talk.

    Unfortunately, I can't think of many more women in the eclipse community, although I'm sure there are quite a few more :-). (Please add more in the comments or in your blog). Only about 2% the people in most open source communities are women. This makes me sad. Why does gender matter? There have been many discussions at eclipse with respect to diversity in terms of company representation among committers. Diversity in terms of gender is important too. Diversity lets us see things from an different viewpoints and that's healthy for our community.

    In Canada, over 60% of new medical school graduates are women, so obviously it's not an issue with women not having the prerequisites to get into engineering and CS. They simply don't choose to enroll in these degrees. Some of the research regarding this issue can be found here.

    So what are steps that we can take as a community to ensure that the committer class of 2020 has more women?
    • Talk to your the kids in our your life about working in open source and how much you enjoy it. Computer science is often perceived as a solitary pursuit, when we all know the truth is that the work is very collaborative. Emphasize the community aspects, and how we all work together to solve problems.
    • Be a mentor. There are many kids in the community who don't have a exposure to a role model who works in the high tech sector. Therefore, they don't know anything about the industry other than the stereotypes. There are many opportunities to volunteer at local schools for career day, helping kids learn to read, or develop math skills. Some companies such as IBM sponsor a camp each year to expose middle school girls to science and technology. Volunteering at this camp allowed me to get paid to build Lego Mindstorms robots for a day. I highly recommend it!
    • Throw away gender based stereotypes. Don't assume your 12 year old niece won't want to learn how to compile a kernel, solder a board, or read a book about web design. You have to start them young :-) Don't call young girl who is interested in engineering or science "geeky", "a tomboy" or "weird". It's normal.

    The purpose of Ada Lovelace day was to for female bloggers to write about a women in technology who inspired them. Here's a list of some other people that have inspired me even though I haven't met them in person.
    • Valerie Aurora was one of the first female Linux kernel committers. She is an expert on Linux file systems and writes on women in open source and building communities.
    • Kathy Sierra is the co-creator of the O'Reilly's excellent Head First line of books. She is also a well known speaker, instructor and blogger who has written on how create users who are passionate about your products.
    • Scienceblogs is an aggregator for scientists who blog about new research in geology, climatology, medicine, math, physics and many other new frontiers of science. Many of the female bloggers write about how they are breaking down gender stereotypes in the ivory towers of academia. Very interesting!

    Read more...

    Slicing and dicing the p2 way

    >> Tuesday, March 03, 2009

    We've traditionally used the packager to craft the features that are available on our download page from the signed bundles in our master feature. However, the packager isn't very efficient because it unzips all of the input feature to the packaging process, regardless of the bundles that are actually needed. Also the packager can be confusing for new users to configure. The packager also doesn't include metadata in the resultant zips. As my neighbour Pascal says, metadata matters. We'd like for people to reconsume the metadata that we create during the build instead of recreating it, possibly with errors.



    In the M6, the p2.mirror task in the org.eclipse.equinox.p2.repository.tools bundle was introduced. This task provides a much more flexible and efficient mechanism for slicing and dicing bundles from a repo into a smaller repo. For instance, to assemble the components of the Equinox SDK from a repo, this command will do the trick:

    <p2.mirror source="file://${buildRepo}">
    <destination kind="metadata" location="file://${featureTemp}" name="My Equinox repo" format="file://${buildRepo}" />
    <destination kind="artifact" location="file://${featureTemp}" name="My Equinox repo" format="file://${buildRepo}" />
    <iu id="org.eclipse.equinox.feature.group" version="" />
    <iu id="org.eclipse.equinox.source.feature.group" version="" />
    <slicingOptions platformFilter="true" includeOptional="false" includeNonGreedy="false" followStrict="true" />
    </p2.mirror>

    This resulting content will include all the bundles associated with the org.eclipse.equinox.feature.group and org.eclipse.equinox.source.feature.group IU's as referenced in the content.jar. The version of these IU's is not specified which means that the highest version will be selected from the repo. Conversely, you could specify a version. The platformFilter="true" means that all platform specific fragments associated with this iu will be provisioned.



    A simple and fast slicer. For the low low price of $19.95. Wait, it's free. Try it out today.

    For more information, see

    bug 262083

    bug 265208

    Read more...

    Implementing composite repos in your build

    >> Friday, February 13, 2009

    As DJ mentioned, composite repository support is included in 3.5. Our I-builds composite repo looks something like this...

    compositeArtifacts.jar
    compositeContent.jar
    I20090210-0950/
    I20090211-0900/

    with a directory for each child repo such as
    ls I20090211-0900/
    artifacts.jar
    binary/
    content.jar
    features/
    plugins/
    sdkinstaller.properties

    Composite repositories allow you to identify which bundles are associated with each build in the repo. This allows you to cleanup your repositories more easily and avoid the wrath of the webmasters. How is this implemented in the build? As a neigbourhood committer would say, let me tell you a story......



    One of the most important steps in our build is to create a master feature of all bundles and features that can sign it at the foundation in a single step. Once this feature is available, we extract it to a source location and publish metadata to a repo.

    <p2.generator source="${reposource}" compress="true" append="true" flavor="tooling" metadataRepository="file:${repo}" artifactRepository="file:${repo}" metadataRepositoryName="${p2.repo.name}" artifactRepositoryName="${p2.repo.name}" publishArtifacts="true" p2OS="linux" mode="incremental" />

    The next step is to run some packaging steps, then use the mirror application task to mirror the metadata and artifacts from the repo to the child repository. Why not just copy the bundles from the repo to the child repository instead of using the artifact.mirror task? One of the things that we want in our build is consistency. If a bundle in build A has the same version and qualifier as the bundle in build B, it should have the same content. Inconsistent bundles are scary.


    How can a bundle have the same name but be different?

    • New compiler changes the byte code
    • New certificate changes the manifest
    • Conditioning process changes the timestamp in the manifest
    • A bundle has a compile error because of one the bundles it depends on has a bug. The dependancy is fixed, but the tag of the original bundle remains unchanged.

    So we mirror the metadata and the artifacts instead of a simple copy.

    <property name="buildRepo" value="${updateSite}/${buildId}">
    <mkdir dir="${buildRepo}">
    <p2.metadata.mirror source="file:${repo}" writemode="append" destination="file:${buildRepo}">

    The artifact.mirror task has the baseline argument which refers to the existing composite repo. This means that the jars in the baseline repo will be used if the same jar exists both the baseline and the source repo. The new jars won't be used. This replicates the experience that a user will have when updating to the latest build. If the same jar already exists in their install, they won't download a new bundle.

    <p2.artifact.mirror source="file:${repo}" baseline="file:${updateSite}" destination="file:${buildRepo}">
    <copy todir="${buildRepo}">
    <fileset file="${eclipse.build.configs}/../../extras/sdkinstaller.properties">
    </fileset>

    The next step is to create the composite repository
    <p2.composite.artifact.repository.create location="file://${updateSite}" name="${p2.repo.name}" compressed="true" failonexists="false">
    <p2.composite.metadata.repository.create location="file://${updateSite}" name="${p2.repo.name}" compressed="true" failonexists="false">

    and add the child repository (current build) to the composite repository. The compositeArtifacts.jar and compositeContent.jar will have entries for the child repositories.

    <children size='2'>
    <child location='http://download.eclipse.org/eclipse/updates/3.5-I-builds/I20090210-0950'/>
    <child location='http://download.eclipse.org/eclipse/updates/3.5-I-builds/I20090211-0900'/>
    </children>


    When we run the p2 director to provision the build zips, we provision from the composite repository.

    Consistent metadata and artifacts ensures happiness all around.




    This solution isn't perfect yet. I'll also be using the p2 team's comparator to provide warnings for certain scenarios . See bug 263272

    Read more...

    Eclipse 3.5 M5 now available

    >> Tuesday, February 03, 2009

    The 3.5M5 Eclipse and Equinox builds, as well as the new and noteworthy have been replicated to eclipse.org. The milestones p2 repo has the M5 bundles available.

    M5 wasn't without drama. Late Friday afternoon, the UI team asked for a rebuild toward 3.5M5 to include a new splash screen. How could this go wrong? A rebuild with one file change. What could be simpler?

    I woke up on Saturday morning confident that the build would be ready to promote. Instead of quality time with my laptop I'd be able to spend time snowshoeing with my favourite mathematician. Unfortunately, this was not to be. The build was missing the eclipse executables (eclipse.exe, eclipsec. exe etc). Hmm, I don't think this is a cosmetic bug. One file change and the executables went away? This seems like overkill.

    When running the p2 director to provision the bundles for the build from the repo, there were signing errors in the .logs. I ran a test build with signing. The build had the same error. I ran a test build without signing. The executables were back. I ran jarsigner -verify on the bundles on a signed build. The junit 3.8.2 bundle had a signing error in the manifest. Interestingly enough, Denis had changed the signing certificate on eclipse.org on Friday.

    At this point I suspected the problem was that the Orbit bundles were signed by the old certificate and the new bundles that we had just compiled in our build were signed in the old certificate. We resign the Orbit bundles in our build. (I've opened this bug to avoid this in the future).

    Martin Oberhuber and David Williams ran a Orbit build on Sunday night which was signed by the new certificate. I ran a test build with this new build. The executables were back! Monday morning I promoted an Orbit build to Stable, and started a new build toward M5. Andrew Niefer determined that the root cause of the problem was that the junit bundle had a nested jar with a bogus signature. The end was near of this milestone saga! I could finally start releasing my patches that I had prepared for M6.

    I was pinged on IRC by a certain PDE committer from Austin who will remain nameless. "Kim? Is there another build toward m5?" This PDE bug was approved by the PMC for inclusion in M5 yesterday afternoon. Another rebuild.



    Here are the bugs with the details.

    Promote 3.5M5
    Certificate will expire within six months.
    JUnit jars should exclude nested jars from signing and packing
    [jarprocessor] Should we be signing nested jars?

    I'll celebrate the release of M5 with a cookie. Enjoy.

    Read more...

      © Blogger template Simple n' Sweet by Ourblogtemplates.com 2009

    Back to TOP