Improving Deployments

Introduction / Problem Statement customers with Large Data Volumes (LDV’s as they are called) can experience prolonged deployments as their Apex test classes run over millions of records. Since my organization has recently reduced our deployments time from over sixty minutes to less than ten I thought I’d share some of the tricks we used.  To give some background on this I’ll share some stats about our org (without actually naming it).  We have 2,142 active users, 920,334 characters of Apex code, 11 Batch Apex classes, and currently use 460.1 GB of our 488.3 GB total storage(we pay extra for storage), and our Account object has 10,682,959 records and growing.


The first thing you’ll need to do is figure out which classes are taking so much time to run.  To do this you’ll need to [Run All Tests] in your production (or in a full copy sandbox if possible) and look at the output to determine which test classes need improvement.

In the output each test class will have next to it the number of milliseconds it took to complete so you’re looking for the ones with the execution time in the hundreds of thousands (123,456 milliseconds is roughly 123 seconds, or 2 minutes).  Search in the execution log for those long running classes and start to look through SOQL statements for long execution times.  Search for SOQL_EXECUTE_BEGIN and compare the time to the next statement of SOQL_EXECUTE_END.  When you see a SOQL query that takes a long time to run you can find out which class is calling it and make adjustments based on one of the options below.

How Apex Test Classes Work

When you run a test class (or all tests) the platform considers the entire transaction to be a single unit of work and (in database terms) rolls back the entire transaction once the test run completes whether it’s successful or not.  This means that all DML statements(insert, update, delete, undelete) are reversed after tests are run.

What causes tests to fail?  If there is (a) an uncaught exception during execution, or (b)a System.assert() fails, or (c)you are deploying to a production org and you don’t have 75% test coverage.  That’s all that come to mind right now.  If you have others please tell me and I’ll amend this.


Data Housekeeping.  If your test classes have to search through Lead records from  three years ago, you might consider archiving that data.  Tell guys in reporting that you want to store this “valuable” information in their data warehouse.  It’ll help them feel important.  Then you can purge those old records out of Salesforce and gain from the reduction of rows your test classes evaluate.  You may want to consider this housekeeping to be a part of your regular maintenance.

Know when you’re testing.  In one of the recent Salesforce releases a new feature was added that makes your classes aware that they are running in test context.  This magic method is Test.isRunningTest() and it allows you to add clauses to your dynamically built SOQL like “order by createdDate desc limit 5”.  My orgs deployment engineer is the one who suggested this and it’s one of the key contributors to speeding up our deploys.

Add indexes on crucial fields.  Another key to our success in reducing deployment time was to add an index on a field we used in one of our Batch Apex classes that was part in the “where” clause.  The index we added was on our Account object which has over 10M rows.  “How do I add an index to a field in Salesforce?” you ask.  Edit the field and mark it as an External Id.  This effectively puts an index on the field in the background.  Use this sparingly, though, because there’s a limit on the number of fields per object you can mark as an External Id.

What not to do

I’ve heard of developers deleting all the records in an object, then inserting specific test records during testing.  Since we know all of the DML operations will be rolled back after the test completes this would seem to be a harmless operations, but if you have 200k Lead records it takes a long time to delete them and to undelete them when you’re done.  So don’t do this!  Be smarter about your test records.  Create them all with a LeadSource of TEST, and in your Apex class check the “Test.isRunningTest()” method and add a ” and LeadSource = ‘TEST'”.


Hopefully this post has give you some ideas on how to speed up your deployments.  The people responsible for running the smoke tests and deployment validations really appreciate getting to bed before midnight and you have better things to do with your time than wait for test classes to finish.

What’s Next

I’ve been considering writing a blog on something we use called TestDataFactory.  A class whose sole responsibility is creating random sets of data for our test classes.  Interested?


About mdnorton

I'm a software developer by day, martial artist by night. I dig writing code, getting some exercise, then kicking back with my home brewed beer. Nunchuck skills, yeah, I got 'em. @NortonMD
This entry was posted in Bookmark the permalink.

2 Responses to Improving Deployments

  1. Osama says:

    Nice post mate!

    Yes we have a class which generates test data for us according to our business process.

    It includes data for Users, Accounts, Contact and Opportunities and some custom objects.


  2. Very interested to hear about your TestDataFactory, particularly if you’d be willing to contribute some bits of it to the open source SmartFactory for!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s