Optimization of equipment. Server Virtualization - Systems To Optimize Cost Buy Any Configuration Selection of Parameters and Cost Selection For the price and calculation Technical optimization solutions for servers

Optimization of database infrastructure and virtual environments

Increase the database infrastructure performance in the current state and receive recommendations for further optimization using cloud services.

SERVER OPTIMIZATION project is relevant in cases:

To increase server performance, you can use several ways, but the best is optimization.

Operating System Optimization (FreeBSD)

• Transition to 7.x. It is useful for multi-core systems, as you can use the new ULE 3.0 Scheduler and Jelloc. If you apply the Legacy 6.x system and it does not cope with the loads, then it's time to make a transition to 7.x.
• Transition to 7.2. Allows you to increase KVA, optimize the SYSCTL default and apply SuperPages. The new FreeBSD 8.0 is already preparing, which will help significantly increase productivity.
• Transition to AMD64. It makes it possible to increase the volumes of KVA and Shared Mem more than 2GB. It is necessary to create conditions for the development of the server, because the database is constantly increasing and require large sizes.
• Unloading the network subsystem FreeBSD will help optimize the server. This process can be made in two stages: Tuning Ifconfig parameters and SYSCTL.CONF / LOADER.CONF settings. At the preparation stage should be checked network card. Drivers from Yandex will help increase the speed due to the involvement of multiple threads, they are often used for multi-core processes. For a third-rate network card better decision will become polling'e. The latest updated version of the FreeBSD 7 tuning will help solve the task.
• FreeBSD and a huge number of fileswonderful thanks to the caching of file names in the directory. The search for a hash table will help you quickly find required file. Although maximum amount The memory is about 2mb, you can increase it while VFS.UFS.DIRHASH_MEM allows you to.
• SoftUpdates., gjournal andmount options. - These are new terabyte screws that have excellent performance. When the power is disconnected, their FSCK will take a lot of time, so you can use softwareupdates or manufacture journaling via Gjournal.

Front Optimization (NGINX)

This species can be attributed in premature optimization, although it will help to increase the general Response Time site. Among standard optimizations should pay attention to reset_timedout_connection; Sendfile; TCP_NOPUSH and TCP_NODELAY.

• Accept Filters. - This is a technology that makes it possible to transmit information from the kernel to the process in case of new data or receiving a validating HTTP request. These filters will help to unload the server with a huge number of connections.
• Caching NGINX is characterized by flexibility, and is made from FastCGI or from Proxy Backen. Each can be smart to use caching in its project.
• AIO. It is very useful with some specific loads on the server, because it saves the Response Time, while the number of wakes is reduced. New versions of NGINX make it possible to use the AIO tandem with Sendfile.

Optimization backend

• APC. - This is a framework that allows you to reduce the load due to the caching of the compiled code in the OP. APC Locking should be updated, as it can brake and instead of APC, many begin to apply Eaccelerator. It is worth replacing the locking on Spinlock or Pthread Mutex. The APC Hints value should be raised with a huge amount of files. Filephp files or when caching is frequent in APC User Cache. APC Fragmentation is a sign that you apply APC is not intended. It cannot independently delete TTL or LRU entries.
• PHP 5.3. It will help to increase productivity growth, so it is worth updating the PHP version, although the list of deplecated functions can scare many.

Database optimization

Ideas for improvement mysql works There are a lot on the Internet, because each web project is sooner or later faced with the limitations of the amount of memory, disk or processor. therefore simple solutions It will not help to cope with the problem, it is worth paying more time to profileers (Dtrace, Systemtap and Oprofile), as well as to use a large number of additional software. It is necessary not only to be able to use indexes to use indexes, to produce their sorting and grouping, but also know how it all functions inside MySQL. You also need to know the advantages and disadvantages of different Storage Engine, understand the Query Cache and Explain.

There are several ways to optimize MySQL, and even without codes changes, because half the server tuning can be carried out in semi-automatic mode using the TUNINGPRIMER, MYSQLTUNER and MYSQLSLA utilities.

• Transition to 5.1. Gives many advantages, among which it is worth highlighting the optimizer optimizer, Partitioning, InnoDB Plugin and Row based Replication. To speed up the site, some extremals are already tested by version 5.4.
• Transition to Innodb. Gives many advantages. It is compatible with ACID, so any operation is performed using just one transaction. It has Row-Level Locking, which makes it possible to simultaneously read and record many streams isolated from each other.
• Built-in Mysql - Query Cache It is quite difficult for understanding, so many users use it irrational or disconnect. For him no longer means better, so you should not bring this subsystem to the maximum. Query Cache is parallel, as a result, when using more than eight processes, it will only slow down the entire process, and not help reduce the site loading time. The contents of this subsystem, which relates to a specific table, is canceled by change in this table. This means that Query Cache gives positive result Only when using competently compiled tables.
• Indexes can be harmful both for SELECT (in their absence) and for Insert / Update (if unnecessary). The index that is no longer used is still the memory and thereby slows down changes. To cope with this problem, you should use a simple SQL query.

PostgreSQL

The Postgres system is quite a versatile, because it refers to the Enterprise class and does Skype work perfectly on it, but at the same time it can be installed even on mobile phone. Among the 200 parameters available, 45 of them are the main and are responsible for tuning.

On the Internet you can find a lot useful information Tuning Postgres. But some articles are already outdated, so you should be repeated from the date of publication and pay attention to the information where the Vacuum_mem key is used, or in new versions of Maintenance_mem. Advanced programmers will be able to find a lot of high-quality treatises, then we will list only those foundations that will help the usual user to improve their project.

• Indexes PostgreSQL is always in the first place, while MYSQL always occupy recent positions and this can be explained by the fact that PostgreSQL indexes have enormous features. The programmer must fine-oriented in such indexes, and to know when and what should be used as Gist, Gin, Hash and B-Tree, as well as Partial, Multicolumn and On Expressions.
• pgbouncer And its alternatives must first be installed on the server with the database. Without the presence of connections bullet, each request creates a separate process that is used rAM. It seems that nothing is terrible, but when creating more than 200 connections, even a very powerful server hardly copes with the processing of information. PgBouncer helps to cope with this problem.
• pgfouine is an indispensable programSince it can be bolded to call the analogue of MySQLSLA on PHP. In Tandem with Playr, it can optimize requests in difficult conditions on staging servers.

Database unloading

To optimize the database and increase its performance, it should be used as little as possible.

• SphinXQLyou can use as a MYSQL server. To do this, you just need to create Sphinx.conf, as well as entries for indexer in CRON and switch to another base. With these actions, there is no need to change the code. The transition to SphinxQL will help increase the speed and quality of the search, as well as forget about Myisam and FTS.
• Non-rdbms storage Allows you not to apply the relational database. You can stop your choice on Hive or Oracle. Database KEY-VALUE Due to its speed applies samples from relational bases For further caching. Owners large projects On PHP, can use excellent OPCode Cache for storing all Custom data. With it, it is possible to reliably save even changes in global value, because they occupy little space and practically do not take memory, as well as the sampling speed to significantly increase. If for a large project a block of global change to record only one machine, then the traffic grows, and it begins to slow down. To solve this problem, it is necessary to store global variables in Opcode Cacher or make cloning variables across all servers and in the Consistentcy Hashing algorithm to register exceptions.
• Coding Active database unloading methods are applied. It is worth noting, UTF-8 is an excellent choice, but in Russian it takes a lot of space, so for a single-speaking contingent, you should first think about the rational use of the encoding.
• Asynchrony will help reduce the response time of the application or site, and also significantly reduce the load on the server itself. Batch requests are produced much faster than familiar single. For huge projects, you can use RabbitMQ, ApacheMQ or Zeromq messages, and only CRON can be used for small.

Additional optimization applications

• SSHGUARD or its alternative It is standard practice for SSH. Anti-bruthfors help create reliable server protection from bots attacks.
• Xtrabackup. From Percona is a wonderful tool for MySQL backup, which has a lot of settings. But the ideal solution Yet it is worth named clones in ZFS, because they are created very quickly, and to produce the database, it is enough to change the paths to the files in the muscle configuration. Clones allow you to restore the system from scratch.
• Mail transfer to another host Allows you to save traffic and IOPS if your server just fall asleep spam.
• Integration with third-party software It will help to optimize MySQL server. For example, you can use the SMTP / IMAP bundle to exchange messages, which will not take a lot of memory. To create a chat, it is enough to use the base of the Jabber server with the JavaScript client. These systems that are created on the basis of adapters to finished products are distinguished by the excellent possibility of scaling.
• Monitoring is a very important component, because it is impossible to optimize something without detailed analysis. It is necessary to follow the performance metrics, free resources and delays, this will help Zabbix, Cacti, Nagios and other tools. Web Performance Test allows you to calculate the download speed of the site or project, so it helps a lot when monitoring. When configuring the server Performance, remember that only thorough analysis will help eliminate all problems that have arisen and optimize.

Did not understand half of the written - not trouble.

The need to optimize the server operating system

Optimization allows, firstly, to optimally configure the PC and thereby achieve maximum performance of the computer, the most complete implementation of it functionality. Secondly, optimization is necessary to maintain a PC, like any device. Ignoring periodic, and in some cases and regular optimization (scanning, cleaning, defragmentation hard disk...) At best, to reduce the speed of the computer, at worst - even to the loss of data.

4. Update Operating system Unix

1. In the bottom right side of the panel, we see the update manager icon. Click on it with a mouse

Figure 20.

2. Like any, respecting yourself and their users, modern system, Linux Mint 14 will ask you to enter an administrative password to make changes to the system. Enter it and click on OK:

Figure 21.

3. The check of repositories will follow the presence of new versions of programs and packages for Linux Mint 14:

Figure 22.

This process will end with the opening of the update manager with a list of updated packages:

Figure 23.

All updated packages are divided in levels, so I think will remind you about these levels is not superfluous

Figure 24.

4. After we chose everything that you wanted to update (or left all items noted), click on the "Install Updates" button:

Figure 25.

5. Linux Mint 14 update manager will notify and show information about those updates that will be installed.

Figure 26.

After you agree, the process of obtaining and updating Linux Mint 14 to the current state

Figure 27.

After the update is completed at the bottom of the manager, you will see this message.

Figure 28.

This means that your system does not require updating and all programs are relevant in it.

Organization of workplace

To improve the workplace, it is necessary to take the following measures:

1. Place the monitor so that its upper point is right in front of your eyes or higher, which will allow you to keep your head straight and eliminate the development of the neck osteochondrosis. The distance from the monitor to the eye should be at least 45 cm;

2. The chair must have a back and armrests, as well as such a height, at which legs can be firmly standing on the floor. The acquisition of a height-adjustable armchair will be ideal, in this case the back will allow you to keep your back straight, armrests will allow you to relax your hands, the correct position of the legs will not interfere with blood circulation in them;

3. The location of frequently used things should not lead to long-staying in any curved position;

4. The workplace lighting should not cause glare on the monitor screen. You can not put a monitor next to the window, so that you simultaneously see and screen and what is outside the window;

5.. Whether working with the keyboard, the angle of folding hands in the elbow must be direct (90 degrees);

6. When working with the mouse, the brush should be straight, and lying on the table as far as possible from the edge.

During work, do not forget about regular breaks for recreation. In the sanitary rules and norms, numerous complexes of eye exercises are given to improve cerebral circulation, to remove fatigue from the shoulder belt and hand, from the body and legs.

Prevention of respiratory diseases includes wet cleaning and ventilating the room. Working at the computer, follow the lighting, posture, take breaks - all this will help increase the ability to work and relieve you from serious diseases. In order to properly organize the workplace, it is necessary to ensure optimal conditions for visual perception in the working area. The most important elements should be placed in the optimal visibility zone.

In the production premises, the optimal working conditions are determined mainly in combination with humidity and air movement. The optimal conditions are the temperature of 20-220 C, the relative humidity of air 30-60%, the speed of its movement is not higher than 0.2 m / s. For the working area, these parameters are determined with the time of the year, the category of work, the amount of heat released in the room. The main causes of the development of fatigue and reduce human performance are noise and vibration. They irritate hearing organs, cause human fatigue and can lead to functional changes in the central nervous system, the circulatory system, etc. Various physical and chemical factors act simultaneously, while their mutual influence on a person can enhance or weaken.

Recently, a request for help in setting up a selected server to operate the online store to 1C-Bitrix. The reason for the appeal is the slow work of the site.
We looked at the site - indeed, some pages are loaded for more than a minute !!!. The first thing that occurred to the site looking to the site is the non-optimal work of the component developed by another developer. But not as much ..

Source data: Server on Xeon - 2GB memory, RAID. OS - FreeBSD. Bus - business.

Well, let's try at least somehow fix the situation ...
Immediately make a reservation that this article is not an instruction for working with a module, just a real case from the life of the module. Maybe someone will be useful.

After the audit, the following main problems were revealed:
1. You must install a PHP accelerator to the server.
2. On early / Price / Large problems with the "NVISIONS: MENU.SECTIONS" component, a query is generated which is processed almost a minute - this is the main reason for the long page loading, as well as a large load on the server.
3. The database (687 recording requests per second is very small) the problem may be in the server configuration. It is necessary to translate the table in InnoDB and configure Innodb
4. The file system is not very fast, it can be the hardware features of the server (for example RAID), but in principle, with such speed, the site should work well
5. The problem is in the site template (there are no existing links (a)) it is necessary to remove it - it takes a lot of resources.
6. You must configure the two-level architecture on the server (static content to give through NGINX), this will significantly reduce the load on aPACHE server, Stabilizes memory consumption on loads, therefore will speed up the work and increase the reliability of the project as a whole.

Let us analyze the performance of the 1C-Bitrix performance module:

From the picture, the problems with the database server are clearly visible, most likely not the optimality of the settings, because The server is dedicated.
Also a suspiciously low number of file operations.


Explicit problems with code or components on the /Price/index.php page
Suspiciously a great time generation /bitrix/urlrewrite.php - look next:

Yeah here it is a source of problems: there is a link to a non-existent image in the template, it generates 404 errors, and causes Apache to process this error and generate a full-fledged page.

The same problem affects all site pages associated with a problem pattern:


And here are the problem components on the page:


The menu component disables caching.
Total Pages:

Well, here is a brief analysis. How convenient the performance module suggests "where problems find". We will proceed to eliminate the problems:

Added a picture to which the link was, it was added a picture, and did not delete the links, because There were many links, including in the components of third-party developers. Also on this page turned off the problem component of a third-party developer (NSVision: Menu.sections), because Its appointment is not clear. (after shutdown, nothing has changed)
Result:


URLREWRITE.PHP is now unbearable on every hit.



As we can see the speed of work increased by 2 times (!).

Going further:
Install Eaccelerator. Here I do not describe how the accelerator is installed, because This information, if necessary, can always be found on the Internet.







Result after installing Eaccelerator: Two more multiple productivity growth.

Going further: We optimize the database (Transfer to Innodb and optimize settings)


As can be seen from the performance of the performance module, the speed of the database has increased significantly
Generally total performance After optimizing the database remained not treason, possibly due to slow work file System.

Update:
Recommendations of the productivity module.
Listening to the instructions of the module, turn off the "Open_Basedir" parameter, because The server is highlighted only under our project, we mean that safety as a whole is not disturbed.

Result:

Result as they say, there is

It remains to rewrite the "crystal" components and the project will fly.

Also installed and configured NGINX as a proxy server for Apache. Pictures do not bring, because The numbers practically did not change. But according to the submitted assessment, the page began to load a couple more times faster.

The template is still generated long enough (the generation time is almost like the system kernel) - apparently, the code of the previous developer is not optimally written. Disassemble someone else's code, there is no time nor budget, no desire. It's easier, faster and cheaper to write your code from scratch.

In general: Performance module is a very useful and convenient tool for debugging project work and server. For which thanks to his developers.

P.S. I personally have a little experience in working with Linux. With FreeBSD, closely met in the first. Surprised that after installing some of the Confruit. Files are generally empty (for example, MySQL). I pleased the ease of installation of software from "ports".