My FreeRADIUS server is slow. What's wrong?

It's probably your database

If you are encountering performance issues with FreeRADIUS, the first thought is usually to blame FreeRADIUS. In fact, it’s always the database which is the source of a system slowdown. Well, maybe not always, but 99 times out of a 100, it’s the database.

It is extremely unlikely that FreeRADIUS is the culprit when you notice slow performance. Here’s how we know.

We have run tests running the server with a minimal configuration, i.e. no database, and minimal processing rules. These tests can easily reach 40,000-50,000 packets per second. But when we add a database such as SQL, the performance drops significantly, to around two thousand packets per second.

Given that FreeRADIUS alone can handle 40K pps, and FreeRADIUS with a DB can only handle 2K pps, the question is do we blame FreeRADIUS, or the DB for the slow performance? In our experience, the database is almost always the performance bottleneck.

We refer to a “database” here, but in practice it can be any external system. For example, shell scripts which are executed when a packet is received (don’t do this), or an external REST API, or anything which is not FreeRADIUS.

Here are some ways you can manage your database(s) to avoid performance problems later on.

Make sure that it’s fast

If the database takes more than about a millisecond to reply to FreeRADIUS, then it’s too slow. If the database takes seconds to reply, then your RADIUS ecosystem will quickly melt down.

Don’t let the database get too big

The more data there is in the database, the slower it will perform. Queries need to look through more records before responding, and writing new records slows down as your disk space fills up.

As part of your regular system maintenance, we recommend implementing a database cleanup strategy.

In the simplest case, this simply means deleting older records which are no longer relevant. Sometimes, organizations are wary about permanently deleting data for a variety of reasons. In that case, you should consider moving older records to a separate database that is not used as part of your RADIUS infrastructure. Which brings us to the next suggestion.

Strategically split the database

The RADIUS server needs access to some data (read and/or write) in order to do its job. But sometimes there is data in the database which the RADIUS server doesn’t need. In those situations, splitting the database is a good plan.

The criteria is that if the RADIUS server doesn’t access a particular record in the DB, then that record should be moved out of the “RADIUS” DB and into another “non-RADIUS DB”. This process usually happens for things like accounting data for user sessions. The user sessions can be split into “online” sessions, and “historical” sessions. If your local process is to keep six months of user sessions, then those sessions should be in a separate database, or a separate database table. Splitting the data in this way ensures that the tables used by RADIUS are small and fast.

Use the right database for your needs

Different databases have unique strengths and weaknesses that can make them ideal in some situations, but underperforming in other scenarios. For instance, SQL databases are very efficient for making complex queries. This makes them a good choice for databases that will used for accounting queries. On the other hand, Redis performs very well in scenarios that require a high volume of database write actions. This makes Redis a good choice for tracking user sessions, or for IP pool management.

In situations with complex requirements, it may be a good strategy to split your database needs across several different databases.

Use database indexes strategically

Database indexes are a useful way to improve query performance. However, there is a performance cost to writing index information. Using unnecessary indexes can double or triple the amount of data which needs to be updated for every database write! And we all know, the more work a database has to do, the slower it gets.

The balance between the performance gains for database queries vs the performance hits for writes depends entirely on your specific use cases. We recommend following a few key principles when designing your database index approach.

  • Use indexes
  • Not too much (no more than 10)
  • Only the ones you will actually use. In other words, don’t create indexes “just in case” they might be useful someday.
  • If you’re not using an index, delete it. (But make a DB backup first!)

Resource your database for the worst case scenario

A successful system administrator is a pessimistic one. As a general best practice, we recommend designing your entire RADIUS infrastructure for the worst possible scenario.

A worst case scenario for your authentication database is that parts of your network will go down for some unforeseen reason. When it comes back up, all of your users will be trying to authenticate at the same time. In order to prepare for this eventuality, the authentication database should not be tested against the expected common case. It should be tested against the worst case.

If the authentication database performs perfectly well in the common case, but falls ober when all of the users try to authenticate at the same time, you may end up facing a series of cascading failures, not to mention angry customers.

Here’s the bottom line, if you suspect that your RADIUS server is being slower than it should be, take a look at your databases. Chances are that’s where your problem is, and your performance problem can be solved with database optimization techniques.

Need more help?

Network RADIUS has been helping clients around the world design and deploy their RADIUS infrastructure for 20 years. We specialize in complex systems and have seen pretty much every variation and problem out there. If you want help from the people who wrote FreeRADIUS, visit our quote page to contact us for a consultation.

Read more...

related articles