In a CentOS 5.5 host, we were requested to upgrade openssh to its latest version. Here are steps I took to quickly do the upgrade. You may like to compile it from source or can take my way of installing it from some repository. Checking existing verison shows 4.3p2: $ ssh -v OpenSSH_4.3p2, OpenSSL 0.9.8e-fips-rhel5 [...]
Архив тега ‘Security’
Quickly upgrade ssh/openssh in CentOS Linux to latest 5.5 version (перепечатка)
Strategies for detecting a compromised Linux server (перепечатка)
There are few things which will rattle systems administrators more than a compromised server. It gives you the same feeling that you would have if someone broke into your house or car, except that it's much more difficult (with a server) to determine how to clean up the compromise and found out how the attacker gained access. In addition, leaving a compromise in place for an extended period can lead to other problems:
- your server could be used to gain access other servers
- data could be stolen from your server's databases or storage devices
- an attacker could capture data from your server's local network
- denial of service attacks could be launched using your server as an active participant
The best ways to limit your server's attack surface are pretty obvious: limit network access, keep your OS packages up to date, and regularly audit any code which is accessible externally or internally. As we all know, your server can still become compromised even with all of these preventative measures in place.
Here are some tips which will allow you to rapidly detect a compromise on your servers:
Abnormal network usage patterns and atypical bandwidth consumption
Most sites will have a fairly normal traffic pattern which repeats itself daily. If your traffic graph suddenly has a plateau or spikes drastically during different parts of the day, that could signify that there is something worth reviewing. Also, if your site normally consumes about 2TB of traffic per month and you're at the 1.5TB mark on the fifth day of the month, you might want to examine the server more closely.
On the flip side, look for dips in network traffic as well. This may mean that a compromise is interfering with the operation of a particular daemon, or there may be a rogue daemon listening on a trusted port during certain periods.
Many compromises consist of simple scripts which scan for other servers to infect or participate in large denial of service attacks. The scans may show up as a large amount of packets, but the denial of service attacks will usually consume a large amount of bandwidth. Keeping tabs on network traffic is easily done with open source software like , , or .
Unusual open ports
If you run a web server on port 80, but netstat -ntlp shows something listening on various ports over 1024, those processes are worth reviewing. Use commands like lsof to probe the system for the files and network ports held open by the processes. You can also check within /proc/[pid] to find the directory where the processes were originally launched.
Watch out for processes started within directories like /dev/shm, /tmp or any directories in which your daemons have write access. You might see that some processes were started in a user's home directory. If that's the case, it might be a good time to reset that user's password or clear out their ssh key. Review the output from last authentication logs to see if there are account logins from peculiar locations. If you know the user lives in the US, but there are logins from various other countries over a short period, you've got a serious problem.
I've used applications like and in the past, but I still prefer a keen eye and netstat on most occasions.
Command output is unusual
I've seen compromises in the past where the attacker actually took the time to replace integral applications like ps, top and lsof to hide the evidence of the ongoing compromise. However, a quick peek in /proc revealed that there was a lot more going on.
If you suspect a compromise like this one, you may want to use the functionality provided by rpm to verify the integrity of the packages currently installed. You can quickly hunt for changed files by running rpm -Va | grep ^..5.
Keeping tabs on changing files can be a challenge, but applications like and good ol' can save you in a pinch.
Summary
We can all agree that the best way to prevent a compromise is to take precautions before putting anything into production. In real life, something will always be forgotten, so detection is a must. It's critical to keep in mind that monitoring a server means more than keeping track on uptime. Keeping tabs on performance anomalies will allow you to find the compromise sooner and that keeps the damage done to a minimum.
is a post from: Major Hayden's blog.
Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.
Quickly disable ssh version 1 in Linux (перепечатка)
This is going to be very short post SSH v1 is not very safe and if you are looking to pass your site/server for PCI compliance then you must disable it. Don’t worry it is too easy to do. Open /etc/ssh/sshd_config file and disable version 1: $ vi /etc/ssh/sshd_config find line: #Protocol 2,1 and remove [...]
Безопасность при использовании directadmin (перепечатка)
Немного о безопасности сервера при использовании такой относительно популярной панели как .
Совсем недавно этот своеобразный набор софта стал причиной взлома нескольких клиентских серверов.
При установке Directadmin, вне зависимости от ваших ответов, ставит в /var/www/html целый ряд индийского быдлокода, который является бомбой замедленного действия:
ls -l /var/www/html -rw-r--r-- 1 root root 44 Jul 29 2010 index.html lrwxrwxrwx 1 root root 44 Jul 29 2010 phpMyAdmin -> /var/www/html/phpMyAdmin-3.3.5-all-languages drwxr-xr-x 11 webapps webapps 4096 Jul 29 2010 phpMyAdmin-3.3.5-all-languages lrwxrwxrwx 1 root root 19 Jul 29 2010 roundcube -> roundcubemail-0.3.1 drwxr-xr-x 10 webapps webapps 4096 Jul 29 2010 roundcubemail-0.3.1 lrwxrwxrwx 1 root root 19 Jul 29 2010 squirrelmail -> squirrelmail-1.4.21 drwxr-xr-x 16 webapps webapps 4096 Jul 19 2010 squirrelmail-1.4.21
Все это (и phpmyadmin и squirrelmail и roundcube) славится обилием публичных эксплойтов, то и дело появляющихся с выходом новых версий.
Примечательно что в дефолтных опциях он еще пополняет этот список такими вещами как Atmail и неведомой хренью с доставляющим названием UebiMiau.
Кроме того, дефолтный DocumentRoot апача смотрит в /var/www/html/, а значит весь этот потенциально дырявый софт очень удобен… для сканеров уязвимостей:
bash# HEAD http://1.2.3.4/roundcube/ 200 OK Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Connection: close Date: Mon, 28 Feb 2011 23:14:00 GMT Pragma: no-cache Server: Apache/2 Vary: Accept-Encoding,User-Agent Content-Type: text/html; charset=UTF-8 Expires: Thu, 19 Nov 1981 08:52:00 GMT Client-Date: Tue, 01 Mar 2011 03:12:00 GMT Client-Response-Num: 1 Set-Cookie: roundcube_sessid=e5057173542fd5756ff78e1a9a4615b9; path=/ X-Powered-By: PHP/5.2.14
Короче говоря, ваш сервер стоит просто и ждет пока его сломает самый ленивый бот, не говоря уже о профессиональном хакере.
Что делать?
Вариант 1.
rm -rf /var/www/html/*
Вариант 2.
В /etc/httpd/conf/httpd.conf найти и заменить
<Directory /var/www/html> ... -- Allow from all ++ Deny from all
А лучше оба вариант сразу.
Другой отличительной особенностью Directadmin является сервер FTP Proftpd. Создатели этого чудо-сервера так же отличаются в написании безопасного софта, работающего под рутом с открытым портом.
По указанной ссылке красуется уязвимость, дающая злоумышленнику рутовый доступ при отправлении специального пакета.
По этому за proftpd так же нужен глаз да глаз. Делается это примерно так
cd /usr/local/directadmin/custombuild ./build update ./build proftpd
После апдейта нужно проверить не доставил ли он в /var/www/html вакханалию, упомянутю в начале статьи и в случае положительного результата, снести.
Так же в последних версиях DirectAdmin был замечен FTP сервер Pure-FTPd. Рекомендую выпилить Proftpd и запилить Pure-FTPd.
Solar Designer on Owl, OpenVZ and stuff: things you ached to know even if you didn't realize you did (перепечатка)
Openwall GNU/*/Linux recently has released . (yeah, you can call it Owl,too, which stands for Openwall Linux) is afloat around ten years, and it's an active and evolving. One of the neat features of Owl 3.0 is OpenVZ support. That was exactly the item which we got excited about, so we asked Solar (and he agreed kindly!) for a short interview.
— When someone is trying to find out what do you do, they mostly come across that you developed , currently lead and participate in different open source security projects. Perhaps there are some new fancy things you're busy with, could you tell us a bit about it?
— This is nothing fancy, but besides the things you mentioned, we (some folks on the Openwall team) spend plenty of time on related client-facing work, including setting up and maintaining Owl/OpenVZ systems remotely, with lots of additional software running on top of them. This is one reason why our Live CDs include SSH remote access, after trivial initial configuration by someone with physical access or over IP-KVM. Then we can proceed with installation starting with partitioning of the hard disks while working over SSH. Besides providing income, which is much needed to fund Owl development and our other activities, this also ensures that we're testing our stuff under real-world usage conditions, and it provides stability for the project (we won't unexpectedly “lose interest” in it one day; we're going to continue to need updates for those systems that we're responsible for ourselves).
As to fancier things, I have plenty of potential project ideas, but too little time. So I mostly have to limit my activities to work related to projects already started. You mentioned that I “developed” John the Ripper, which I did, but it shouldn't just stay “developed” for many years more, it needs to evolve further, and there's demand for that (such as from penetration testing firms). This is about coming up with algorithms to produce more optimal candidate password streams, as well as about having them reflect new kinds of weak passwords that people start to use. This is also about efficient use of new CPU types, parallel processing, distributed processing, and use of GPUs. I did only a little bit of work on some of these things lately, being busy with lots of other stuff, but this is something I need to get back to and have specific ideas on.
A luckier project in 2010 was , our proactive password/passphrase strength checking tool set (PAM module, library, command-line programs). Unlike John the Ripper, which audits password hash files, passwdqc does not permit weak passwords to be set in the first place. Dmitry V. Levin (of ) and I made numerous enhancements to passwdqc: implemented new features, more effective checks (and tested those checks on thousands of real-world passwords), and at the same time made passwdqc even more portable. Proactive password/passphrase strength checking with passwdqc (applied when a user changes their Unix password) is now supported on Linux, FreeBSD, DragonFly BSD, OpenBSD, Solaris, and HP-UX.
— I've read through Owl release notes, and even that frightening «Weee SUID ftw!/O, no, no SUID» , so what's so unique in your distro except no SUID (thanks to Openwall guys) and OpenVZ on board (feeling proud of ourselves)?
— It's a combination of things that make our distro unique, although a few things are also unique on their own: the lack of SUID programs in a usable default install, and syslogd credential passing (anti-spoofing). Some other things that make Owl special are:
* Proactive source code reviews of security-critical components of third-party software that we package.
* Security hardening changes in many packages, including some relatively uncommon changes (e.g., environment variable sanitization in glibc, Debian-generated weak key blacklisting in OpenSSH, privilege separation added to telnetd).
* Self-contained yet small: although the distro is small, it contains everything needed to easily rebuild itself from source.
* The included build environment is additionally capable generating ISOs and OpenVZ container templates («make iso» and «make vztemplate»).
* A single CD contains a live system, installable packages, the installer program, as well as full source code and the build environment.
* Old-fashioned text-only installer, simple scripts and configuration files, and reduced bloat overall. The system does not try to outsmart its administrator.
* RPM'ed default kernel, yet manual custom kernel builds are conveniently supported as well (only a few cumulative patch files are applied in the package).
* A good selection of small yet handy sysadmin and diagnostic tools, such as dmidecode, hdparm, ltrace, mdadm, mtree and nc (our ports from OpenBSD), Nmap (and Ncat), pciutils, smartmontools, strace. Of course, most of these are also available for/on mainstream distros, yet given Owl's purpose as a good «base system» for servers it is worth mentioning that we include all of these and more in our otherwise small set of packages, as well as in a default install.
— How big approximately is the user base?
— There's no «Owl user» registration mechanism, so I have no numbers on the size of our user base. It «feels» small, yet sufficient for us to continue with the project. Besides, much of the software that we develop as part of Owl we're also making available separately, and many of our patches, etc. are reused by other distros. For example, our «tcb» suite, which allows for the «passwd» command to work without being SUID, is also reused by ALT Linux's distributions and by Mandriva. This increases its user base a lot.
— What are the most common use cases for the Owl?
— Most common use cases? For us, that's OpenVZ «host systems», as well as OpenVZ container templates built with Owl as the «base system» and with extra software added (sort of «virtual appliances» that we make and install ourselves). We (or rather our client companies) mostly use this to provide «advanced» web hosting.
Here's a project built on top of OpenVZ and Owl, , which is a website development platform, involving multiple backend servers internally, with features such as «one-click» installs of popular web apps and cloning of existing «virtual hosts» (such as for the development/QA/production lifecycle).
We've also experimented with building Owl-based virtual machine hard disk images for a specific project, where the VMs would be run on end-users' desktop/laptop computers. Owl worked well for this purpose.
Since Owl's OpenVZ support is still relatively new (introduced into Owl-current a year ago, but only included in a stable release with Owl 3.0), we're hoping that more people and companies will make use of it in various ways, including in «virtual appliances» that they'd release. These could be VM images that also run OpenVZ containers, or they could be OpenVZ container templates or a mix of both.
Finally, we're aware of a McAfee (now Intel) hardware appliance product that is built upon an Owl-derived system (without OpenVZ, though). We'd be happy to see more hardware appliances make use of Owl as well, including of its OpenVZ integration.
We'd be happy to help more companies make and maintain Owl-based and Owl-derived systems, appliances, etc. Much of this work can be outsourced to us if desired. I think that hosting providers offering Linux-KVM VPS hosting may be interested in making Owl one of their standard distro options. They may specifically advertise the ability for a customer to create multiple OpenVZ containers inside such a VPS, which may provide an extra selling point for their offering vs. the competitors'. I am not aware of hosting providers doing this yet, although I am aware of a user of Owl having tried this out on his own and he was pleased with the result. Thus, I am mentioning this not so much as an answer to your question, but rather to encourage such use by hosting providers.
— I usually pay attention to mascots 
looks like an owl near the boundary post/booth, it's really cool! Was it the first idea? Or there was a contest for a best picture?
— I'm glad that you like it. There was no formal contest. The Owl project logo that we currently use was based on one of several sketches contributed by Gleb Todosov, who also created our CD jewel case artwork. We used the latter up through our 2.0 release, and you can still see a portion of it in use on the first slide in our .
— We're excited that you support OpenVZ in your distro and would like to dig into details. Why did you decide to do that? I guess you tried OpenVZ before, what was your previous experience? What did you use it for?
— Personally, I got interested in container-based virtualization in 1999 or thereabout (although I didn't know the concept would be known under this name). My first exposure to Virtuozzo was during SWsoft's public beta testing, which I think was in 2000. It was possible to register for a free trial account, which I did, and I got onto an SWsoft-hosted free trial system running a 2.4.0-test kernel with early and unreleased Virtuozzo patches. I ended up finding a vulnerability and reporting it to SWsoft.
Fast forward to 2005, the OpenVZ project was about to be announced to the public, and Openwall was contracted by SWsoft to perform a security audit of OpenVZ, which I personally worked on in close coordination with the OpenVZ developers. This was a pleasant experience, and the code quality was better than what I was used to seeing. Most of the issues that were discovered during the audit got fixed promptly. During the audit, I made my own test install of OpenVZ (for fuzzing, etc.) on top of an Owl system on a machine dedicated for the purpose.
OpenVZ was released in 2006. As far as I can tell by looking at file timestamps now, it was August 2006 when we slowly started to deploy it on top of some of our clients' Owl systems — replacing the kernels and adding vz* tools, but keeping the Owl userland on what became OpenVZ host systems. After a long while, we had plenty of systems running Owl and OpenVZ together, and we also had Owl-based userlands running inside «VEs» (the word «container» was not in use in OpenVZ context yet). At some point, we needed to introduce more sysadmins to the team and we also needed more systems of this kind installed to satisfy our clients' needs. It proved painful to explain all the little details (hacks) of making things work right.
We had the idea of integrating OpenVZ into Owl before — I even briefly mentioned it during a 2006 interview — but the difficulty mentioned above might have been the straw that finally made us do it, despite that it meant temporarily giving up some of our kernel hardening patches (not reimplemented for 2.6/OpenVZ kernels yet).
— What do you like in latest releases (already tried VSwap?)? What do you not like (I myself can hardly imagine that after there is no need to set all the UBC manually and get the blind headaches there is something left to improve, but I'm sure I need a fresh look on those things), so is there anything you find obscure or inconvenient?
— Of your «latest releases», we're only making (a lot of) use of the RHEL5 branch kernels («testing» and/or «stable» on different occasions), with some additional changes (we make our own builds). I guess that you were referring to your newer branches. Unfortunately, we have not tried those out yet. This means that we haven't tried VSwap yet, even though this is a very welcome and anticipated feature (and change from the older UBC model). We're likely to start moving to your RHEL6 branch kernels in Owl-current this year, and we're likely to use them in our next major release (Owl 4.0). We'll be sure to provide feedback as we do this.
As to further improvements that we'd like to see in OpenVZ, it's security hardening, primarily against Linux kernel vulnerabilities that might turn into «container escape» ones in OpenVZ-patched kernels. One fairly obvious enhancement would be to be able to set a container to 32-bit only, 64-bit only, or mixed. The first two settings would disallow the other type of syscalls, thereby significantly reducing the exposure of kernel interfaces to possible attacks from the container's system. Another Owl developer tells me that this specific enhancement was also suggested upstream (LXC, I guess).
— Is there anything you would like to add or tell to millions of OVZ blog readers?
— Feedback, please! We need more feedback on Owl; join the owl-users mailing list and post there, please. Successes, failures, comments, opinions, requests, all kinds of feedback is welcome. Also, please contribute to our . So far, it appears that most people who use Owl successfully don't bother contacting us («no need»), those who run into minor issues and solve/workaround the issues on their own also don't bother notifying us («no need» or so they think, wrongly), and most of those who run into trouble or are unhappy with Owl for whatever reason either notify individual Owl developers privately or (possibly) give up without telling us of the problem. If you use Owl (or have tried or considered to), please do your part to help correct this «feedback problem».
Solar Designer on Owl, OpenVZ and stuff: things you ached to know even if you didn't realize you did (перепечатка)
Openwall GNU/*/Linux recently has released . (yeah, you can call it Owl,too, which stands for Openwall Linux) is afloat around ten years, and it's an active and evolving. One of the neat features of Owl 3.0 is OpenVZ support. That was exactly the item which we got excited about, so we asked Solar (and he agreed kindly!) for a short interview.
— When someone is trying to find out what do you do, they mostly come across that you developed , currently lead and participate in different open source security projects. Perhaps there are some new fancy things you're busy with, could you tell us a bit about it?
— This is nothing fancy, but besides the things you mentioned, we (some folks on the Openwall team) spend plenty of time on related client-facing work, including setting up and maintaining Owl/OpenVZ systems remotely, with lots of additional software running on top of them. This is one reason why our Live CDs include SSH remote access, after trivial initial configuration by someone with physical access or over IP-KVM. Then we can proceed with installation starting with partitioning of the hard disks while working over SSH. Besides providing income, which is much needed to fund Owl development and our other activities, this also ensures that we're testing our stuff under real-world usage conditions, and it provides stability for the project (we won't unexpectedly “lose interest” in it one day; we're going to continue to need updates for those systems that we're responsible for ourselves).
As to fancier things, I have plenty of potential project ideas, but too little time. So I mostly have to limit my activities to work related to projects already started. You mentioned that I “developed” John the Ripper, which I did, but it shouldn't just stay “developed” for many years more, it needs to evolve further, and there's demand for that (such as from penetration testing firms). This is about coming up with algorithms to produce more optimal candidate password streams, as well as about having them reflect new kinds of weak passwords that people start to use. This is also about efficient use of new CPU types, parallel processing, distributed processing, and use of GPUs. I did only a little bit of work on some of these things lately, being busy with lots of other stuff, but this is something I need to get back to and have specific ideas on.
A luckier project in 2010 was , our proactive password/passphrase strength checking tool set (PAM module, library, command-line programs). Unlike John the Ripper, which audits password hash files, passwdqc does not permit weak passwords to be set in the first place. Dmitry V. Levin (of ) and I made numerous enhancements to passwdqc: implemented new features, more effective checks (and tested those checks on thousands of real-world passwords), and at the same time made passwdqc even more portable. Proactive password/passphrase strength checking with passwdqc (applied when a user changes their Unix password) is now supported on Linux, FreeBSD, DragonFly BSD, OpenBSD, Solaris, and HP-UX.
— I've read through Owl release notes, and even that frightening «Weee SUID ftw!/O, no, no SUID» , so what's so unique in your distro except no SUID (thanks to Openwall guys) and OpenVZ on board (feeling proud of ourselves)?
— It's a combination of things that make our distro unique, although a few things are also unique on their own: the lack of SUID programs in a usable default install, and syslogd credential passing (anti-spoofing). Some other things that make Owl special are:
* Proactive source code reviews of security-critical components of third-party software that we package.
* Security hardening changes in many packages, including some relatively uncommon changes (e.g., environment variable sanitization in glibc, Debian-generated weak key blacklisting in OpenSSH, privilege separation added to telnetd).
* Self-contained yet small: although the distro is small, it contains everything needed to easily rebuild itself from source.
* The included build environment is additionally capable generating ISOs and OpenVZ container templates («make iso» and «make vztemplate»).
* A single CD contains a live system, installable packages, the installer program, as well as full source code and the build environment.
* Old-fashioned text-only installer, simple scripts and configuration files, and reduced bloat overall. The system does not try to outsmart its administrator.
* RPM'ed default kernel, yet manual custom kernel builds are conveniently supported as well (only a few cumulative patch files are applied in the package).
* A good selection of small yet handy sysadmin and diagnostic tools, such as dmidecode, hdparm, ltrace, mdadm, mtree and nc (our ports from OpenBSD), Nmap (and Ncat), pciutils, smartmontools, strace. Of course, most of these are also available for/on mainstream distros, yet given Owl's purpose as a good «base system» for servers it is worth mentioning that we include all of these and more in our otherwise small set of packages, as well as in a default install.
— How big approximately is the user base?
— There's no «Owl user» registration mechanism, so I have no numbers on the size of our user base. It «feels» small, yet sufficient for us to continue with the project. Besides, much of the software that we develop as part of Owl we're also making available separately, and many of our patches, etc. are reused by other distros. For example, our «tcb» suite, which allows for the «passwd» command to work without being SUID, is also reused by ALT Linux's distributions and by Mandriva. This increases its user base a lot.
— What are the most common use cases for the Owl?
— Most common use cases? For us, that's OpenVZ «host systems», as well as OpenVZ container templates built with Owl as the «base system» and with extra software added (sort of «virtual appliances» that we make and install ourselves). We (or rather our client companies) mostly use this to provide «advanced» web hosting.
Here's a project built on top of OpenVZ and Owl, , which is a website development platform, involving multiple backend servers internally, with features such as «one-click» installs of popular web apps and cloning of existing «virtual hosts» (such as for the development/QA/production lifecycle).
We've also experimented with building Owl-based virtual machine hard disk images for a specific project, where the VMs would be run on end-users' desktop/laptop computers. Owl worked well for this purpose.
Since Owl's OpenVZ support is still relatively new (introduced into Owl-current a year ago, but only included in a stable release with Owl 3.0), we're hoping that more people and companies will make use of it in various ways, including in «virtual appliances» that they'd release. These could be VM images that also run OpenVZ containers, or they could be OpenVZ container templates or a mix of both.
Finally, we're aware of a McAfee (now Intel) hardware appliance product that is built upon an Owl-derived system (without OpenVZ, though). We'd be happy to see more hardware appliances make use of Owl as well, including of its OpenVZ integration.
We'd be happy to help more companies make and maintain Owl-based and Owl-derived systems, appliances, etc. Much of this work can be outsourced to us if desired. I think that hosting providers offering Linux-KVM VPS hosting may be interested in making Owl one of their standard distro options. They may specifically advertise the ability for a customer to create multiple OpenVZ containers inside such a VPS, which may provide an extra selling point for their offering vs. the competitors'. I am not aware of hosting providers doing this yet, although I am aware of a user of Owl having tried this out on his own and he was pleased with the result. Thus, I am mentioning this not so much as an answer to your question, but rather to encourage such use by hosting providers.
— I usually pay attention to mascots looks like an owl near the boundary post/booth, it's really cool! Was it the first idea? Or there was a contest for a best picture?
— I'm glad that you like it. There was no formal contest. The Owl project logo that we currently use was based on one of several sketches contributed by Gleb Todosov, who also created our CD jewel case artwork. We used the latter up through our 2.0 release, and you can still see a portion of it in use on the first slide in our .
— We're excited that you support OpenVZ in your distro and would like to dig into details. Why did you decide to do that? I guess you tried OpenVZ before, what was your previous experience? What did you use it for?
— Personally, I got interested in container-based virtualization in 1999 or thereabout (although I didn't know the concept would be known under this name). My first exposure to Virtuozzo was during SWsoft's public beta testing, which I think was in 2000. It was possible to register for a free trial account, which I did, and I got onto an SWsoft-hosted free trial system running a 2.4.0-test kernel with early and unreleased Virtuozzo patches. I ended up finding a vulnerability and reporting it to SWsoft.
Fast forward to 2005, the OpenVZ project was about to be announced to the public, and Openwall was contracted by SWsoft to perform a security audit of OpenVZ, which I personally worked on in close coordination with the OpenVZ developers. This was a pleasant experience, and the code quality was better than what I was used to seeing. Most of the issues that were discovered during the audit got fixed promptly. During the audit, I made my own test install of OpenVZ (for fuzzing, etc.) on top of an Owl system on a machine dedicated for the purpose.
OpenVZ was released in 2006. As far as I can tell by looking at file timestamps now, it was August 2006 when we slowly started to deploy it on top of some of our clients' Owl systems — replacing the kernels and adding vz* tools, but keeping the Owl userland on what became OpenVZ host systems. After a long while, we had plenty of systems running Owl and OpenVZ together, and we also had Owl-based userlands running inside «VEs» (the word «container» was not in use in OpenVZ context yet). At some point, we needed to introduce more sysadmins to the team and we also needed more systems of this kind installed to satisfy our clients' needs. It proved painful to explain all the little details (hacks) of making things work right.
We had the idea of integrating OpenVZ into Owl before — I even briefly mentioned it during a 2006 interview — but the difficulty mentioned above might have been the straw that finally made us do it, despite that it meant temporarily giving up some of our kernel hardening patches (not reimplemented for 2.6/OpenVZ kernels yet).
— What do you like in latest releases (already tried VSwap?)? What do you not like (I myself can hardly imagine that after there is no need to set all the UBC manually and get the blind headaches there is something left to improve, but I'm sure I need a fresh look on those things), so is there anything you find obscure or inconvenient?
— Of your «latest releases», we're only making (a lot of) use of the RHEL5 branch kernels («testing» and/or «stable» on different occasions), with some additional changes (we make our own builds). I guess that you were referring to your newer branches. Unfortunately, we have not tried those out yet. This means that we haven't tried VSwap yet, even though this is a very welcome and anticipated feature (and change from the older UBC model). We're likely to start moving to your RHEL6 branch kernels in Owl-current this year, and we're likely to use them in our next major release (Owl 4.0). We'll be sure to provide feedback as we do this.
As to further improvements that we'd like to see in OpenVZ, it's security hardening, primarily against Linux kernel vulnerabilities that might turn into «container escape» ones in OpenVZ-patched kernels. One fairly obvious enhancement would be to be able to set a container to 32-bit only, 64-bit only, or mixed. The first two settings would disallow the other type of syscalls, thereby significantly reducing the exposure of kernel interfaces to possible attacks from the container's system. Another Owl developer tells me that this specific enhancement was also suggested upstream (LXC, I guess).
— Is there anything you would like to add or tell to millions of OVZ blog readers?
— Feedback, please! We need more feedback on Owl; join the owl-users mailing list and post there, please. Successes, failures, comments, opinions, requests, all kinds of feedback is welcome. Also, please contribute to our . So far, it appears that most people who use Owl successfully don't bother contacting us («no need»), those who run into minor issues and solve/workaround the issues on their own also don't bother notifying us («no need» or so they think, wrongly), and most of those who run into trouble or are unhappy with Owl for whatever reason either notify individual Owl developers privately or (possibly) give up without telling us of the problem. If you use Owl (or have tried or considered to), please do your part to help correct this «feedback problem».
5 steps to secure your Linux server (перепечатка)
How would you ensure security of your production Linux Server? Should you can be happy with default configuration there in place or there’s are things which we must implement for enhancing security aspects? Of course, Yes. Here I’m writing 5 steps which I usually take to tighten security in Server. This doesn’t means these are [...]
On kernel exploits and OpenVZ user beancounters (перепечатка)
Yesterday a guy with his name written in Cyrillic letters («Марк Коренберг») and a @gmail.com email address a kernel exploit to the Linux kernel mailing list (aka LKML). This morning one brave guy from our team tried to run it on his desktop — and had to reboot it after a few minutes of total system unresponsiveness.
The bad news are the exploit is pretty serious and causes Denial of Service. It looks like most kernels are indeed vulnerable.
The good news is OpenVZ is not vulnerable. Why? Because of .
The nature of exploit is to create an unlimited number of sockets, thus rendering the whole system unusable so you need to power-cycle it to bring it back to life. Now, if you run this exploit in an OpenVZ container, you will hit the numothersock beancounter limit pretty soon and the script will exit. This is an except from /proc/user_beancounters after the run:
cat /proc/user_beancounters
uid resource held maxheld barrier limit failcnt
.....
numothersock 9 360 360 360 1
.....
As you can see, current usage is 9, while peak usage is 360, same as limit. Finally, failcnt is 1 meaning there was once a situation when the limit was hit.
I went further and set numothersock limit to 'unlimited', and re-run the exploit. The situation is much worse in that case (don't try it at home, you've been warned), system slows down considerably, but I was still able to login to the physical server using ssh and kill the offending task from the host system using SIGTERM.
/proc/user_beancounters look after the second run:
uid resource held maxheld barrier limit failcnt
111: kmemsize 1237973 14372344 14372700 14790164 80
.....
numothersock 9 2509 9223372036854775807 9223372036854775807 1
As you can see, even with numothersock set to unlimited, another beancounter, kmemsize, is working to save the system.
Of course, if you set all beancounters to unlimited, exploit will work. So don't do that, unless your CT is completely trusted. Those limits are there for a reason, you know.
On kernel exploits and OpenVZ user beancounters (перепечатка)
Yesterday a guy with his name written in Cyrillic letters («Марк Коренберг») and a @gmail.com email address a kernel exploit to the Linux kernel mailing list (aka LKML). This morning one brave guy from our team tried to run it on his desktop — and had to reboot it after a few minutes of total system unresponsiveness.
The bad news are the exploit is pretty serious and causes Denial of Service. It looks like most kernels are indeed vulnerable.
The good news is OpenVZ is not vulnerable. Why? Because of .
The nature of exploit is to create an unlimited number of sockets, thus rendering the whole system unusable so you need to power-cycle it to bring it back to life. Now, if you run this exploit in an OpenVZ container, you will hit the numothersock beancounter limit pretty soon and the script will exit. This is an except from /proc/user_beancounters after the run:
cat /proc/user_beancounters
uid resource held maxheld barrier limit failcnt
.....
numothersock 9 360 360 360 1
.....
As you can see, current usage is 9, while peak usage is 360, same as limit. Finally, failcnt is 1 meaning there was once a situation when the limit was hit.
I went further and set numothersock limit to 'unlimited', and re-run the exploit. The situation is much worse in that case (don't try it at home, you've been warned), system slows down considerably, but I was still able to login to the physical server using ssh and kill the offending task from the host system using SIGTERM.
/proc/user_beancounters look after the second run:
uid resource held maxheld barrier limit failcnt
111: kmemsize 1237973 14372344 14372700 14790164 80
.....
numothersock 9 2509 9223372036854775807 9223372036854775807 1
As you can see, even with numothersock set to unlimited, another beancounter, kmemsize, is working to save the system.
Of course, if you set all beancounters to unlimited, exploit will work. So don't do that, unless your CT is completely trusted. Those limits are there for a reason, you know.
Accessing Rackspace Cloud Servers and Slicehost slices privately via OpenVPN (перепечатка)
Diagram: OpenVPN to Rackspace Cloud Servers and Slicehost
A recent inspired me to write a quick how-to for Fedora users on using OpenVPN to talk to instances privately in the Rackspace Cloud.
The diagram at the right gives an idea of what this guide will allow you to accomplish. Consider a situation where you want to talk to the MySQL installation on db1 directly without requiring extra ssh tunnels or MySQL over SSL via the public network. If you tunnel into one of your instances, you can utilize the private network to talk between your instances very easily.
There's one important thing to keep in mind here: even though you'll be utilizing the private network between your tunnel endpoint and your other instances, your traffic will still traverse the public network. That means that the instance with your tunnel endpoint will still get billed for the traffic flowing through your tunnel.
You'll only need the openvpn package on the server side:
yum -y install openvpn
Throw down this simple configuration file into /etc/openvpn/server.conf:
port 1194 proto tcp dev tun persist-key persist-tun server 10.66.66.0 255.255.255.0 ifconfig-pool-persist ipp.txt #push "route 10.0.0.0 255.0.0.0" push "route 10.176.0.0 255.248.0.0" keepalive 10 120 ca /etc/openvpn/my_certificate_authority.pem cert /home/major/vpn_server_cert.pem key /home/major/vpn_server_key.pem dh /etc/openvpn/easy-rsa/2.0/keys/dh1024.pem status log/openvpn-status.log verb 3
Here's a bit of explanation for some things you may want to configure:
push— These are the routes that will be sent over the VPN that are pushed to the clients. If you don't use any IP addresses in the 10.0.0.0/8 network block in your office, you can probably use the commented out line above. However, you may want to be more specific with the routes if you happen to use any 10.0.0.0/8 space in your office.server— These are the IP addresses that the VPN server will assign and NAT out through the private interface. I've used a /24 above, but you may want to adjust the netmask if you have a lot of users making tunnels to your VPN endpoint.ca, cert, key— You will need to create a certificate authority as well as a certificate/key pair for your VPN endpoint. I already use on my Mac to manage some other CA's and certificates, but you can use scripts if you wish. They are already included with the openvpn installation.
Build your Diffie-Hellman parameters file:
cd /etc/openvpn/easy-rsa/2.0/ && ./build-dh
Tell iptables that you want to NAT your VPN endpoint traffic out to all 10.x.x.x IP addresses on the private network:
iptables -t nat -A POSTROUTING -s 10.0.0.0/8 -o eth1 -j MASQUERADE
The last step on the server side is to ensure that the kernel will forward packets from the VPN endpoint out through the private interface. Ensure that your /etc/sysctl.conf looks like this:
# Controls IP packet forwarding net.ipv4.ip_forward = 1
Adjusting your sysctl.conf ensures that forwarding is enabled at boot time, but you'll need to enable it on your VPN endpoint right now:
echo 1 > /proc/sys/net/ipv4/ip_forward
Start the openvpn server:
/etc/init.d/openvpn start
If all is well, you should see openvpn listening on port 1194:
[root@lb2 ~]# netstat -ntlp | grep openvpn tcp 0 0 0.0.0.0:1194 0.0.0.0:* LISTEN 2020/openvpn
You'll need to configure a client to talk to your VPN now. This involves three steps: creating a new certificate/key pair for the client (same procedure as making your server certificates), signing the client's certificate with your CA certificate (same one that you used above to sign your server certificates), and then configuring your client application to access the VPN.
There are many openvpn clients out there to choose from.
If you're using a Linux desktop, you may want to consider using the . For Mac users, I'd highly recommend using ($9), but there's also (free).
is a post from: Major Hayden's blog.