I have been busily (spell-check says that IS a real word) working on new features and fixes for MozTrap for a couple months now and we’ve pushed out several improvements that should make your MozTrapping easier (spell-check says that IS NOT a real word, but I still believe in it).
I have added a Release Notes section to the docs. As always, you are free to check out our Pivotal Tracker site to see where we are and what we’re working on now and next. But the release notes is hopefully a little more concise.
Excerpt from the Release Notes for some project highlights. Since I haven’t written a blog post since 1.1, I’ll share all the feature lists going back to 1.2:
- Sharable list links – When you have filtered a list somewhere in the system, you can click the link icon next to the filter field to bring up the url that you can share to show that list. This link honors pagination and all filters. And it can be used in the management area as well as results and in test runs. This can be especially nice if you want to tell a tester to run a specific set of test cases in a run.
- Test Run description while running tests – We added the test run description field to the top of the page while running tests. This field supports markdown, so you can put links and other instructions to your testers in there. This can be especially helpful to add links to creating a new bug in your bugsystem of choice. (You ARE using Bugzilla, aren’t you?) See Run Edit Fields for more info.
- Filtering performance – In some screens, the auto-complete filters were being displayed for every keystroke. Now they always wait till you’re done typing before showing auto-complete options.
- Run activation scalability – Using some new features in Django 1.4 and a couple raw queries, we expanded support for test runs from ~700 cases to several thousand.
- Update active test runs – The new refresh button in the management area will update an active run to newly added or removed test cases. See Refreshing a Run for more information.
- Case import management command – The feature for importing cases would prevent you from importing duplicates, even if you wanted to. So added a param for that. It also accepts a directory of several files instead of just a single file.
- Django 1.4.2 upgrade
- More non-ascii character fixes – Primarily in some views and messages.
- Split-the-work: When you and others are executing the same test run, for the same environment, you’ll see an icon on test cases where another tester has already submitted results. You can still submit your own result if you choose, but this way you don’t duplicate effort, if you don’t want to.
- Test case ordering – As you drag and drop cases in the edit Suite screen, that order will be honored when users run your tests. Same goes for suites of test runs. So, the order will be first by suite, then by case within the suite. There is also a new field in the runtests area where, if you sorted by case name, you can re-sort by order, if you like.
- Performance fix for editing large suites – Scalability fix as thousands of cases had been entered into the system.
- Run Series: See Test Run Series for more info on this new feature.
- Better i18n support – Added more support for non-ascii characters.
I hope you like the new features. Please reply here or visit us on irc.mozilla.org #moztrap with any questions or feedback.
For those just tuning in….
Historically manual test case management at Mozilla used an old tool called Litmus. Litmus was a good tool for its time, but it had several design flaws that made it difficult for us to grow it as the Mozilla QA team’s needs expanded. Today, we are happy to announce Litmus’ replacement, MozTrap. MozTrap is the “better mousetrap” of testcase management tools, making test tracking easier and more efficient, and ultimately more hackable and expandable as our needs change. It helps make our test case management system far more organized and flexible than the older system ever dreamed of being. For example:
- Filtering: Whether you’re looking at a list of products, suites, runs or cases, you’re always able to filter down to the list you’re interested in. You can filter by several fields at the same time to narrow down to just what you want. This includes during test execution. If you’re executing a run with 100 cases, but you only want to execute the ones having to do with “feature X” then filters are a simple, intuitive way to get you there.
- Tagging and Markdown: You can add tags to test cases to make a subset of them quicker to find with filtering. And you can also create rich text descriptions and steps for your test cases with the popular Markdown formatting.
- Test case versions: Products grow, and, therefore, so must test cases. As you add new versions of your products to MozTrap, you will have a version of the test case applied to each one. You can tweak the test case information to match the new product functionality. And for a specific test case, you can browse the history of how the case has changed for each version of the product. This makes it easy to keep newer case steps separate from older ones when you need to test a new version, but still support an older one.
- Environment management: Everyone’s product is a little different with specific environmental factors that apply. It may be a list of browsers, or it may be several spoken languages. It could be operating system, or even processor or hard drive type. The environment management is completely configurable. And you can have specific test cases that apply to only one, or a subset of environments.
- FOSS and up to date: Of course it’s open source, it’s built on Django so deployment is easy, and it incorporates Mozilla’s Persona identity system for login as well as Mozilla’s upcoming App Marketplace.
There’s a lot more to MozTrap than I could explain here. Please check it out!
- You can browse around read-only on our staging site here: https://moztrap.allizom.org/
- Or download it from github and run it locally: https://github.com/mozilla/moztrap/
- Check out the documentation here: http://moztrap.readthedocs.org/en/1.0.X/index.html
- Visit our forum at: https://groups.google.com/forum/?fromgroups#!forum/moztrap
- Feel free to ask questions: irc.mozilla.org #moztrap
- Enter bugs with this link: https://bugzilla.mozilla.org/enter_bug.cgi?component=MozTrap&product=Mozilla%20QA
We hope to hear from you soon,
OK, so I have to eat some crow here. I talked about some new features we had planned for the “end of May” in my last blog post. And then the priorities changed. Most of our dev team was pulled off MozTrap to work on another Mozilla project. So the pace of development has slowed down quite a bit. In the mean time, the Mozilla QA team has begun transitioning over from Litmus to MozTrap for testing and it’s been a great experience. We are getting feature requests and questions that are helping to shape MozTrap. Most of them are questions we answer with, “We have a story for that. It’s on the roadmap.” and some of them are, “Oh, cool. Great idea!”
So, our adjusted feature set for 1.1 sometime near the beginning of July will be a tad more modest, even if they did require a lot of solid work:
- Some minor bug fixes
- Automation Results API feature
- Open Web App support
Most of the features mentioned before will be slated for Q3, 2012. So stay tuned.
Next I’ll talk a bit more about Mozilla QA transitioning to MozTrap and how we’re using it.
William Lachance from the A-Team here at Mozilla is credited with creating this Meme poster for us. I thought it was hilarious.
And on we go with the 1.1 version of MozTrap. We have enough features on our wishlist to take us for a couple more years. And it seems like the list keeps growing the more we get great feedback.
For now, here is the short-range list of features we’re working on for 1.1:
- REST API for external tools to submit results: This comes in 2 parts. The first is the REST APIs we are creating, the second is a python connector to make it easy to interact with that API. The intention is that your automation tool can run tests and submit them as “Automation Bot 3” (or whatever name you choose) and have those results reflected in MozTrap
- Tags on Test Runs: It occurs to us that often times you might like to organize test runs for several “Products” and monitor their progress as a unit. By tagging test runs, you can filter down to just the runs you’re concerned with and get a clear picture on your status.
- Test Case ordering within Suites: At the moment, the order that tests show up when running isn’t set in stone. We realize that often times it’s much easier to execute tests in a particular order.
- Global Product Filtering: Often times, you are a user that is concerned with just one Product that your company offers. By being able to set a filter to only show you that product, you don’t have to keep filtering through cases, suites and runs for other products.
- Sticky Filtering: when you set up some filters on, say, the test cases management list, then edit one of the tests, and come back to the list, your filters are gone. Most often, a user would like those filters to persist. We’re working on it.
- Bulk edit for tags and status: doing these one by one works, but sometimes doing it in bulk is just lots easier.
That’s our list for the immediate future. We are hoping to get these changes out by the end of May. So keep your eyes open!
-Cam – MozTrap Project Manager – Mozilla
My team and I have been working on this product for over a year, and we have finally hit our 1.0 version!
The name? Oh that… Thank you for all the responses to the survey. Seemed like people’s opinions went in quite a few different directions. Some liked short and simple names. Some liked funnier names. Some liked more descriptive names. In the end, we went with something that (in some ways) had all three of those…
MozTrap – Set the trap, catch the bugs. Sure, it’s a play on “Mouse Trap” or perhaps on Mosquito Trap. Goofy, short, easy to spell, and descriptive (after a fashion…). We are really excited about MozTrap hitting 1.0. This is my first big project at Mozilla, and I am having a blast.
I want to give a big shout-out to my amazing team: You guys really kick asteroids and I can’t thank you enough for all your enthusiasm, creativity, tenacity, hard-work and dedication. Carl Meyer, Eric Meyer, Rebecca Billings, Jonny Meyer and Sondra Eisenstat have worked super hard and humored me in my quirks and crazy ideas to fit them into this product we now call MozTrap. You guys really made an awesome push all the way through to the end. From the bottom of my heart, I thank you. And a special thanks to Aakash Desai for getting us rolling on this project and helping guide me into it before heading off to help make Mozillians awesome.
Here are the main important links:
- Source – https://github.com/mozilla/moztrap
- Docs – http://readthedocs.org/docs/moztrap/en/1.0.X/index.html
- Dev site – https://moztrap-dev.allizom.org
So, now on to more features!
We have a big list of wishlist items in our Pivotal Tracker project, and we’re working hard to prioritize them, and move forward. Stay tuned for more updates.