What to do when you get lost in API testing?

If you are not writing tests yet, you should start. Tests will improve the quality of your code, reduce bugs and probably force you to think more about design and quality. Actually, the hardest thing is to start and figure out what type of tests to write. 

So you start googling and you find a lot of test types: unit, integration, acceptance, functional, system, smoke, e2e… Some resources will say you need to have 100% coverage, some will say full coverage doesn’t mean your code is fully tested. Even when you decide what type of tests to use, it can be hard to decide how to test some specific logic in your app. 

Don’t give up! We have some tips we learned along the way and hopefully it will make your life easier.

Don’t try to test it all with just functional tests

This is the number one mistake developers do in testing. So let’s clear that first.

Functional tests in REST Api would mean sending request on one endpoint and asserting its response. If your route has multiple different cases in which domain logic can go, you would need to write separate test for each case to be sure they all work.

This could even work on a very small app, but as soon as your app grows, this will become very hard to maintain and they will be very slow. At some point you will start losing more time on fixing and maintaining tests than actually working on your app. That is not what tests should be about. We learned that the hard way.

The Test Pyramid

Test pyramid is a graph telling us how much testing you should write on each layer. 

Lower layers are faster and that’s why we need to cover as much as possible with unit and integration tests. The more high-level you get, the fewer tests you should have. You will hear different namings for individual layers, but logic behind them is the same.

Unit tests

So let’s start from the beginning — Unit tests. Here’s a definition:

Unit testing is a level of software testing where individual units/components of a software are tested. The purpose is to validate that each separate unit of the software performs as expected. A unit is the smallest testable part of any software.

This means you should test each method in isolation and mock all other methods .

It’s not unit tests, it’s you!

Unit tests are not just good for making sure your code works, but it will also make you do some refactoring. They should be easy to write. If you struggle with testing a particular piece of code, it means your code is badly written and it should be refactored. SOLID principles will help you here. SOLID is a set of design principles for writing clean, elegant and readable code. If you write solid code, unit tests will be a piece of cake.

So basically, with unit tests you make sure each isolated method in your code works as it should. That would mean all the code works as expected, right? Not really. You also need to make sure individual components work ok together. And that’s where integration tests come.

Integration tests

Integration testing is the phase in software testing in which individual software modules are combined and tested as a group.

Good example when unit test is not enough is database query builders. In unit tests you can make sure you called conditions you want in the query, but if your query is more complicated, you can’t really be sure conditions you used will return expected result until you make an actual call to the database. So this is where integration test that runs the query on a test database and asserts expected results should come.

Cases where you need integrations tests:

• Verifying correct integration with 3rd party libraries or APIs
• Verifying two or more modules which have unit tests work as expected together

Functional tests

Functional testing is defined as the testing of complete functionality of some application.

They are much simpler to write in API compared to an application with a user interface. Functional tests for API should consist of sending actual requests to API endpoints and validating the response format. You should test each route for successful and error response.

But don’t over do it, you don’t need a test for each possible error. That should already be covered by unit and integration tests. Here you just want to make sure your route returns correct response format for an error. If your routes have different roles, you should also test security error response.

Here are some helpful tools to make your functional tests easier to write:

• PHP Matcher— Library that enables you to check your response against patterns. You can set which pattern you expect for each field in your response, instead of asserting real values in your database. This way you are able to test only response format and field types without worrying if some string or number changed in your test database.
• Postman tests— you can write your functional tests in Postman which is pretty useful for the frontend team, they will be able to check and try your api endpoints. It will be like interactive API documentation. You can integrate Postman tests with Newman — command line runner which enables you to run tests in command line and integrate them in Continuous integration(CI).
• Faker — library for generating fake data for your test database

Going one step further

Now that you know your code works, you can check its performance.

Load testing

Load testing checks if your api is responding as expected to various servers.

Stress testing

Checking if API performs as expected when receiving large number of requests at the same time.

Do I need to have 100% coverage?

100% coverage doesn’t guarantee your code is fully tested and working. This only means tests used each line of your code, but it sure helps to run the code coverage to see if you missed to cover some parts of your code.

We learned that we don’t benefit from writing tests for simple methods as getters and setters. They take time to write and maintain, especially on big projects.

While tests will eliminate most of the potential bugs, there will still be some edge cases you didn’t think of. If a bug appears, make sure to reproduce it with new tests first, and then fix your code. That way your coverage will improve and you will be sure you covered that case. This can also be a great way to start covering your legacy projects that don’t have tests yet.

Here are some helpful tools to check quality of your code and tests:

• Infection — tool for mutation testing. Mutation testing evaluates quality of existing tests. Mutation testing modifies a program in a small ways and expects your tests to fail which means mutant will be killed. For each passed test, mutant stays alive. Test suites are measured by the percentage of mutants that they kill. Mutation testing provides a testing criterion called the Mutation Score Indicator (MSI). The MSI measures the effectiveness of a test set in terms of its ability to detect faults. You can also integrate it to CI and set minimum MSI required in order for CI to pass.
• PHPStan — PHPStan focuses on finding errors in your code without actually running it. It catches whole classes of bugs even before you write tests for the code. It moves PHP closer to compiled languages in the sense that the correctness of each line of the code can be checked before you run the actual line.
• Continuous integration(CI)— enables you to run your tests on git on each commit. It will prevent you from merging PR-s with failing tests.

Conclusion

It’s easy to get lost in all the things you find about writing tests. The main goal of testing should be making sure your code works and it’s easy to maintain. If you are new to testing, it might seem they take too much time, but don’t give up, once you get a hold of it, it will make your coding life much easier and faster. 🙂