This month, we welcome Guillaume Betaillouloux from Octoperf, a software solution dedicated to performance testing. An opportunity to learn more about their product and its added value for this startup founded in 2016.

 

Hello Guillaume! To start, can you introduce us to Octoperf?

OctoPerf is a company created by four performance lovers four years ago. With the idea of providing a powerful and affordable performance testing tool to as many people as possible.

 

How did the Octoperf adventure start?

We all had about 10 years of experience in our respective jobs, and at some point, we disagreed with our former employers’ business and code quality policies. We really wanted to demonstrate that we could provide a better product for a controlled cost in the long term with efficient code quality control.

 

If we go back to the basics: can you explain what performance testing consists of?

It consists of simulating the arrival of a flow of users on an application to simulate a period of high activity, generally anticipating a marketing event or simply anticipating a production launch.

 

What are your key advantages and strengths?

The use of an open-source product at the heart of the solution (JMeter) while offering a completely revisited UX around this product. This allows heterogeneous teams to cohabit with beginners who used our UI and experienced JMeter users who import their favorite scripts.

 

What are the prerequisites (especially technical) to start working with you?

You must be willing to work on the performance of your applications. OctoPerf can be integrated into all technical environments, either via the SaaS platform or the installation of a private enterprise platform that allows all the company’s collaborators to work together on the tests.

 

Which profiles in a company are in charge of writing these performance tests?

This varies a lot depending on the context. In general, they are rather technical profiles with a good knowledge of the web world, but thanks to OctoPerf, it is possible for more functional profiles to conduct just as realistic tests.

 

Let’s take a concrete case: I am an e-commerce editor, and I don’t have any performance tests in place. Is it the right time for me to use Octoperf?

Yes, our advice is to participate in a product demo to understand the possibilities and then get used to it with the free account that allows you to launch small tests. Then we always spend some time with new customers on a proof of concept session to prepare coherent test scenarios together. This allows you to start with a small license in “one-shot” mode and build your performance testing practice little by little at controlled costs. With of course the goal to regularize these tests in the long run.

 

If I already use solutions like JMeter, what will I gain by going on your platform?

Well, the idea is that we take care of the complexity of JMeter while offering all the features of a commercial solution. You just have to reuse your usual JMeter scripts and upload them in OctoPerf to get the right to launch injectors on the fly and, above all, to get a real-time report which is customizable.

 

Is there maximum scalability that you can test with Octoperf?

We have run tests with several hundred thousand concurrent users from about 400 machines, but this is not a limit, just the largest use case ever tested.

The combination of elasticsearch and the latest SSDs allows us to aggregate incredible amounts of test data in real-time on the database side.

So there is virtually no limit on our side because it is mainly the load injectors started in platforms like AWS or Azure that could be limiting, and on that side, there is room for improvement.

 

What could be the cause of the performance “problems” that Octoperf detects? A non-robust infrastructure? An application not designed to scale?

It’s very variable. The common case is a lack of configuration of the tested servers (default settings on web/app/db servers). But there are many other issues, like forgetting to optimize resources for production or poorly optimized code. This is why it is important to combine performance testing with monitoring tools that can detect the causes of problems or upstream quality measurement tools that can avoid late problems.

At Promyze, we are very sensitive to best development practices. Do you recommend a set of best practices on web technologies that can improve the performance of websites?

We already respect many code quality rules ourselves, well beyond the standard rules. And we have noticed that a “clean” codebase allows incredible agility in the development and evolution of the product since our development team is always composed of 2 people.

It is difficult to give an absolute answer to answer the question since not all contexts are the same. For example, an e-commerce needs a lot of reactivity to follow the marketing and the site is likely to be rewritten every 2-3 years so an absolute quality of the code is not necessarily desirable or realistic.

However, it is important to master these subjects quickly to optimize maintenance costs and technical debt. To use a well-known analogy, code quality is like cleaning the kitchen in a restaurant. It doesn’t pay off in the short term, but it’s essential in the long term, both for the customers and the employees who work there.

The best thing to do is to try and see where you stand about other types of solutions and see how they proceed. Internet is rich in feedback.

The advice that our CTO always gives is the famous KISS (Keep It Simple and Stupid) and to work already with tools that we master well before going on new “cool” or “trendy” technologies that are attractive but still young or sometimes too unstable for professional use.

In case of problems with your platform, do you have a service offer (consulting/training) to help your customers review the architecture of their applications or their infrastructure?

As a software editor, we focus on our business to provide a simple and efficient tool and help our users get the most out of it. We provide OctoPerf training sessions (two days for groups of about 10 people) that allow even novices to approach the subject more serenely.

If our customers need help to optimize their systems, we rely on our network of worldwide partners who can intervene on an ad hoc basis or on a more regular basis to carry out all the tests.

 

We regularly see e-commerce sites “down” during key events in the sector (release of a flagship product, first day of sales, …). Can we really avoid these 100% KOs  or should we try to reduce the risks as much as possible?

We can never protect ourselves 100% from problems, but many companies protect themselves at 0%, which leaves a lot of room for improvement. Of course, performance testing is a way to reduce this risk very strongly, but only if it is practiced under good conditions. We see a lot of companies doing extremely simplistic tests. This will give them an important perceived risk coverage, but in fact, they remain vulnerable.

That’s why we insist on the training, the methodology, and the advanced capabilities of OctoPerf for our users.

 

Do you work with clients from all sectors?

Yes, that’s why we use JMeter at the heart of OctoPerf. The number of supported protocols and the existing plugins really cover the essential testing needs of all types of companies. And above all, the JMeter community is composed of hundreds of thousands of testers, so the amount of content that can be found online is much higher than what any other paid tool can offer.

 

Is Octoperf an exclusively SaaS solution, or do you offer an On-Premise version?

We have both. This allows us to cover all use cases. The idea is to use SaaS as much as possible for simplicity, sometimes with an on-premise injection agent, to get around IP whitelisting or DDOS protection concerns. But when the need for confidentiality is too important, we can install our own servers and have the test data in a secure space.

Some of our customers have an enterprise version and connect it to their Azure/AWS account to take advantage of on-the-fly load injectors.

What are your goals for 2021, and can you tell us about the new features you are working on?

OctoPerf is growing, we have three new resources that joined us during the year, and we will continue on this path. Our big news is the development of a new product called Kraken and oriented around Gatling, the second most popular open-source solution after JMeter.

For this product, we are testing an open-source model with the addition of paid modules. Currently, everything is available for free in Kraken to allow everyone to get an idea of its potential, and we wish it the same success as OctoPerf.

—-

To find all the information about Octoperf: https://octoperf.com