I have struggled for years to understand what the point of the cgi-bin folder was for which then lead me to research into the whole area and what was the point of this technology.
What is CGI?
- CGI stands for 'Common Gateway Interface'
- CGI is a standard for external gateway programs to interface with information servers such as HTTP servers.
- Or in other words, programs in other languages can be used and utilised via HTTP requests.
- A way to run a server side script (PHP, Perl, Python,...) when a HTTP request comes.
- This allows Binaries and scripts of other programming languages to be run such as Python and C programs.
- Currently maintained by the NCSA (National Center of Supercomputing Applications).
- CGI has been around long before PHP.
What is the purpose of CGI?
CGI allows the use of scripts and binaries from many other languages and the benefit of this are:
- You can utilise these languages and their inherent feature sets directly from a website.
- Different languages have different abilities
- Programming is not limited to the web-based languages.
`cgi-bin` folder, why do I have this on my web server?
This is the only folder allowed on your server where you can runs CGI applications and is usually created when you install the package `php-cgi`.
How Does it work?
- Place scripts or binaries in the folder /cgi-bin/ with the file extension .cgi, other might work such as .py but I have not tried this.
- Run the script accessing the relevant URL such as https://example.com/cgi-bin/mytest.cgi
- The Script:
- is started
- reads the Stdin (Standard in) and Environmental variables as required
- processes the data
- outputs the response to the Stdout (Standard Out)
- is terminated
- The webserver returns the Stdout as the http response
- The file extension .cgi can be changed if required when setting up the server.
- The .cgi files can be either scripts or binaries.
- A CGI can only external interact externally as follows (but this does not include what its internal routines do):
- Read the Stdin
- Read the Environmental Variables
- Output the Stdout
- When a CGI App is started a single process for it is created and when it has finished executing the process is terminated removing it from RAM etc.
- You can easily upgrade a CGI script by just changing the CGI file.
Running PHP via CGI, FastCGI, PHP-FPM
PHP can be run via any of the following wrappers but this is not the same for CGI Apps
- mod_php
- Old, insecure and slow.
- Never use this.
- This is the original way to run PHP and is here for reference only.
- CGI
- PHP runs as a CGI App
- Opens a single process for each request, runs it and then when finished, is immediately closed.
- FastCGI
- PHP runs as a FastCGI App
- Maintains a pool of workers for running scripts
- FastCGI can use a single persistent process which handles many requests over its lifetime.
- PHP-FPM
- FastCGI Process Manager (FPM)
- FPM (FastCGI Process Manager) is a primary PHP FastCGI implementation containing some features (mostly) useful for heavy-loaded sites.
- Only runs PHP, does not allow for traditional CGI or FastCGI apps.
- This now the recommend method for PHP and is by far the fastest.
- This is know as PHP-FPM rather than just FPM because it only runs PHP.
- The Linux package name is php-fpm
Notes
- CGI is one process for each request and has large overheads.
- FastCGI
- uses a persistent process and requires more settings for this.
- is a bit more complicated because the processes start up at the very beginning of the server.
- FastCGI keeps a pool of scripts running whereas CGI opens a single process, runs it and then when finished is immediately closed. Because the processes are always open there is no overhead of starting up the process any more.
- CGI and FastCGI Apps are not the same, but are similar. FastCGI Apps need extra code to handle persistent states.
- You can run a CGI App under FastCGI but you need a wrapper app.
- CGI and FastCGI are almost always run from the cgi-bin folder
- The file extensions of .cgi and .fcgi are use as appropriate
CGI and FastCGI are getting replaced
These technologies are getting or already have been replaced with proxying and dedicated server modules such as mod_perl and mod_python.
Most programming languages offer their own dedicated servers so it makes sense to have those do the relevant work and then return the results to your web server. Proxying allows a web server to do this invisibly just as CGI and FastCGI have done in the past and allows the resource hit to be offloaded.
Proxying is configured in your Apache Config files.
What about PHP-FPM?
This technology is dedicated to running PHP, and in a sense is probably now a dedicated engine for PHP only.
I cant see this going away anytime soon but it will only ever run PHP.
Virtualmin
I have put my Virtualmin notes here as they are more relevant to this niche feature.
- Virtualmin only supports CGI Apps out of the box. You can manually add FastCGI support.
- Virtualmin installs mod_fcgi, because it is a dependency of the php extension php-fpm which is installed by default.
- mod_fcgi requires you to set the correct handlers in the Apache conf and these are not set by Virtualmin.
- Virtualmin supports CGI Apps out of the box as and has GUI pages for this.
- Your PHP mode does not affect the running of CGI Apps, they are separate.
- Does the php-cgi module include cgi and fastcgi
- no, php-fpm includes FastCGI and PHP-FPM
- php-cgi includes only CGI
- GUI Locations
- Webmin --> Servers --> Apache Webserver --> Global Configuration --> CGI Programs
- Webmin --> Servers --> Apache Webserver --> Existing virtual hosts --> select virtual server --> CGI Programs
- Virtualmin --> Web configuration --> (Configure SSL Website | Configure Website) --> CGI Programs
- Changing the PHP mode does not change whether the server is going to run CGI or FastCGI Apps, it only alters how PHP s run.
- Should ExecCGI be removed from the public_html folder apache directive? - Virtualmin - Virtualmin Community
Q:
- My understanding is that ExecCGI is the command that can enable or disable the running of a CGI.
- In the code below it looks like the apache directive is allowng the enabling of CGI Apps in the root of the public_html folder.
- So my question is, should ExecCGI be removed from the public_html folder or have I missed something, perhaps this is required for PHP to be run?
<Directory /home/example/public_html> Options -Indexes +IncludesNOEXEC +SymLinksIfOwnerMatch +ExecCGI Require all granted AllowOverride All Options=ExecCGI,Includes,IncludesNOEXEC,Indexes,MultiViews,SymLinksIfOwnerMatch </Directory> <Directory /home/example/cgi-bin> Require all granted AllowOverride All Options=ExecCGI,Includes,IncludesNOEXEC,Indexes,MultiViews,SymLinksIfOwnerMatch </Directory>
A:
AllowOverride
means.htaccess
can override the default configuration. The default configuration does not haveExecCGI
.- You can, of course remove it, if you don’t want to allow that to be overriden by
.htaccess
. Historically it was pretty common to run CGI scripts in public_html, too, but much less so these days. - Presumably you allow execution of PHP in
public_html
, if you’ve got any web apps installed. So, if a remote attacker gains ability to write topublic_html
, it’s already over. There would be no need to modify.htaccess
to run scripts, they’d just drop a PHP shell inpublic_html
and be done.
- Does Virtualmin support FastCGI Apps - Virtualmin - Virtualmin Community
- Q: I know Virtualmin supports standard CGI Apps, but does it support FastCGI Apps?
- A:
- Apache (and nginx) support FastCGI. Virtualmin isn’t your web server.
- It generally needs some app-specific configuration, which kinda depends on what language(s) and app server(s) you’re using and how you’re using them. The things Virtualmin knows how to configure with FastCGI it does so (Install Scripts might use FastCGI, for example, and that will get configured). There is no fcgi-bin directory, though, by default, so there’s nowhere you can just drop a FastCGI application and expect it to magically run.
- The necessary Apache module (mod_fcgid) is installed by default, though. mod_fcgid - FastCGI interface module for Apache 2 - The Apache HTTP Server Project
- Best selection of PHP extensions - #13 by Jamie - Virtualmin - Virtualmin Community
- php-fpm is independent of running PHP apps via fcgid+suexec (which uses php-cgi and predates the existence of php-fpm). FPM is an application server for PHP apps that speaks the FastCGI protocol. FastCGI is a protocol, not any specific implementation, and there are several ways to run Apache+PHP apps using the FastCGI protocol.
- You should use FPM, if you can. The implementation is much simpler (from the user perspective), it is much more actively maintained, it is more efficient, etc. And, it provides the same security benefits of suexec.
- Can I remove the system default CGI PHP
- php-cgi has nothing to do with the virtualmin miniserv which runs on Perl.
- The CLI version of PHP has nothing to do with CGI.
- Can I safely remove CGI on my alternative php versions?
- yes
Links
- CGI
- The Magic of cgi-bin - YouTube | Philip Bohun
- An introduction to cgi and why you might want to think about using for your web project.
- This would be the first Video I watched.
- W3 Security FAQ: CGI (Server) Scripts - Discusses security issues with CGI scripts.
- Definition of CGI script | PCMag
- What does CGI script actually mean? Find out inside PCMag's comprehensive tech and computer-related encyclopedia.
- Although CGI scripts are still used, PHP and other languages are more popular. Active Server Pages (ASPs), JavaServer Pages (JSPs) and Java servlets have also been used to tie a request to a database. See PHP, ASP, JSP, servlet and FastCGI.
- Apache Tutorial: Dynamic Content with CGI - Apache HTTP Server Version 2.4 - This document will be an introduction to setting up CGI on your Apache web server, and getting started writing CGI programs.
- The Magic of cgi-bin - YouTube | Philip Bohun
- FastCGI
- General
- FastCGI.com Archives
- This is the Github FastCGI Archives of FastCGI.com site.
- Since fastcgi.com is not available anymore (last backup on wayback machine are at 20/03/2016), I (mcarbonneaux) created this repository and linked website fastcgi-archives.github.io to host the FastCGI information backup.
- FastCGI A High-Performance Web Server Interface | FastCGI.com Archives - This is Technical White Paper.
- Understanding FastCGI Application Performance FastCGI | FastCGI.com Archives - Just how fast is FastCGI? How does the performance of a FastCGI application compare with the performance of the same application implemented using a Web server API?
- Guides/FastCGI - J Wiki - J can be used for FastCGI programming. Here are some considerations and examples.
- FastCGI - Wikipedia
- FastCGI — The Forgotten Treasure - This paper shows developers how high-performance web applications can be implemented quite easily using the FastCGI interface and C++. After a brief introduction to the FastCGI interface, a generic C++ framework for web-based applications using FastCGI will be discussed, including the advantages and disadvantages of this approach.
- FastCGI.com Archives
- Apache mod_fcgid
- mod_fcgid - FastCGI interface module for Apache 2 - The Apache HTTP Server Project - mod_fcgid is a high performance alternative to mod_cgi or mod_cgid, which starts a sufficient number instances of the CGI program to handle concurrent requests, and these programs remain running to handle further incoming requests. It is favored by the PHP developers, for example, as a preferred alternative to running mod_php in-process, delivering very similar performance.
- mod_fcgid - Apache HTTP Server Version 2.5 - Provides for execution of FastCGI applications
- Installing mod_fcgi
- How to Install and Configure FastCGI and PHP-FPM on CentOS 8 | Linode Docs - This guide will show you how to install and configure the mod_fcgid and PHP-FPM protocols for dynamic content generation and processing using Apache on CentOS 8.
- mod_fcgi htacces directives
- CGI | Virtualmin
- This is what my new .conf file looks like (added +ExecCGI + MultiViews and added .cgi Handler)
Options -Indexes +IncludesNOEXEC +FollowSymLinks +ExecCGI +Multiviews allow from all AllowOverride All AddHandler fcgid-script .php AddHandler fcgid-script .php5 AddHandler cgi-script .cgi FCGIWrapper /home/domain/fcgi-bin/php5.fcgi .php FCGIWrapper /home/domain/fcgi-bin/php5.fcgi .php5
- This is what my new .conf file looks like (added +ExecCGI + MultiViews and added .cgi Handler)
- .htaccess - Hiding fcgi script file name from url in django app hosted with fastcgi and apache - Stack Overflow
- CGI | Virtualmin
- General
- CGI vs FastCGI vs PHP-FPM
- How does Fast CGI work? - YouTube - A simple video but explains the difference well
- Fast CGI - YouTube | Philip Bohun
- A quick video describing fast CGI, how it differs from CGI, as well as its pros and cons.
- PHP: CGI and command line setups - Manual | php.net
- By default, PHP is built as both a CLI and CGI program, which can be used for CGI processing.
- PHP CGI and FPM, what are they? - Stack Overflow
- PHP: FastCGI Process Manager (FPM) - Manual | php.net
- php fpm - Differences and dis/advanages between: Fast-CGI, CGI, Mod-PHP, SuPHP, PHP-FPM - Server Fault
- CGI vs FastCGI vs PHP-FPM. CGI: CGI stands for Common Gateway | by Miladev95 | Medium - CGI:CGI stands for Common Gateway Interface.- It is a standard protocol that defines how web servers can interact with external
- CGI
- While CGI was revolutionary in its time and enabled dynamic web content, it has certain limitations, including performance overhead and scalability issues. As a result, modern web development has largely moved away from CGI in favour of more efficient and scalable server-side scripting technologies like PHP, Node.js, and various application frameworks.
- Today, CGI is mostly used in specific cases where compatibility with legacy systems or specialized environments is required. For general-purpose web development, more efficient alternatives are preferred.
- In summary, CGI is the simplest and oldest method for executing server-side scripts but suffers from performance overhead. FastCGI improves upon CGI by introducing a persistent process pool, resulting in better performance. PHP-FPM, on the other hand, is a specific FastCGI implementation tailored for PHP execution, providing superior performance, scalability, and security for serving PHP applications. For modern PHP applications, PHP-FPM is the recommended method for handling dynamic content.
- FastCGI
- FastCGI is widely used in web server configurations to handle dynamic content generation efficiently. Popular web servers like Nginx and Apache support FastCGI, making it a standard choice for hosting dynamic web applications written in languages like PHP, Python, Ruby, and more.
- In summary, FastCGI is an extension of CGI that significantly improves the performance and resource utilization of web applications by employing a persistent application process pool to handle incoming requests.
- PHP-FPM
- Overall, PHP-FPM is a valuable tool for PHP application deployment, providing improved performance, security, and scalability for serving PHP-driven web applications.
- In summary, CGI is the simplest and oldest method for executing server-side scripts but suffers from performance overhead. FastCGI improves upon CGI by introducing a persistent process pool, resulting in better performance. PHP-FPM, on the other hand, is a specific FastCGI implementation tailored for PHP execution, providing superior performance, scalability, and security for serving PHP applications. For modern PHP applications, PHP-FPM is the recommended method for handling dynamic content.
- CGI
- What are CGI and fast CGI - This is part of the Semicolon&Sons Code Diary - consisting of lessons learned on the job. You're in the web-development category.
- Which PHP mode? Apache vs CGI vs FastCGI - Layershift Blog - There are multiple ways to execute PHP scripts on a web server. We’re often asked about the difference between these modes, so here it is!
- Difference between PHP-CGI and PHP-FPM | BaseZap - When running PHP through the web server, there are two distinct options: running it using PHP's CGI, or running it as a PHP-FPM, for the web server.
- What is difference between PHP cli and PHP cgi? - Stack Overflow
- Basically it's a way to run a server side script (PHP, Perl, Python,...) when a HTTP request comes.
- Excellent descriptions.
- CGI, FastCGI and PHP-FPM - Some basic questions - Help! (Home for newbies) - Virtualmin Community
- Q: Is running CGI Apps now considered an old way of doing things?
- A: Generally, yes. For large applications, having them load all their dependencies on every run is wildly inefficient. For tiny little apps that don’t load a bunch of extra junk, it may be fine. e.g. if you just need to serve out one tiny API endpoint and you can write a Perl/Python/Ruby/PHP script that runs in a few milliseconds, then it may make sense to use CGI (but an app server probably isn’t notably worse than CGI in those cases, either…it’d be about balancing CPU vs. memory usage).
- Q: FastCGI - allows CGI Apps to be run in cgi-bin
- A: No. FastCGI can not run CGI scripts. It requires applications built to communicate via the FastCGI protocol, which is not the CGI protocol. A CGI script is not a FastCGI script without a wrapper. We provide a wrapper (fcgiwrap), so you can run CGI applications on systems that don’t have suexec available, but it does not mean FastCGI is running CGI scripts. fcgiwrap runs CGI scripts and communicates with the web server via the FastCGI protocol. It is the application server in this context.
- Q: But, this is a bit misleading: “Virtualmin only supports CGI Apps out of the box. You can manually add FastcGI support.”
- A:
- Virtualmin supports, with full UI support, PHP apps via many execution modes, including FastCGI, and multiple versions of PHP (when installed). Virtualmin also supports setting up proxy rules to any other kind of application server (many app servers do not use FastCGI, they just use HTTP, and that is the most common way to run most apps other than PHP at this point in history). FastCGI is on the way out, too.
- We brought proxy paths support from Pro into GPL just recently to ease that use case. And Virtualmin Pro is about to get app server and container-based app support (where it has some awareness of the app server or docker/podman and configures the proxy rules automatically).
- So, nearly any way you want to run apps in Virtualmin is very easy to do.
- Old Technology
- How to run PERL scripts - #7 by Joe - Help! (Home for newbies) - Virtualmin Community
- You should be moving your apps to a modern execution mode. Plack is probably the easiest way to migrate CGI Perl scripts to something modern. https://plackperl.org/
- Q: Do I need Plack, if I use Apache?
- A:
- CGI is the slowest way to run a web app, so the web server is completely irrelevant in your deployment, so I recommend you use the one that is most commonly used for CGI scripts, which is Apache.
- You don’t need Plack. I was offering you a solution for improving the way you’re executing your apps. Plack is a low-friction path from CGI to running apps under a long-running Perl app server (this saves significant startup time for each request). If you use Plack, you no longer need CGI script support in your web server or a wrapper (because it is the wrapper and it provides a fastcgi interface you can proxy to). If you were to use Plack, you would be able to use nginx. Performance still wouldn’t matter, but at that point I would recommend whichever web server you’re most comfortable with rather than having a relatively strong opinion that Apache is the better choice.
- I don’t know anything about the app you’re running; there’s nothing I can tell you about it. But, I know how CGI works with Apache and how it doesn’t work with nginx (unless you wrap it). So, if you’re running CGI scripts, you should go the path of least resistance, unless you have a good reason to do otherwise (performance is not a good reason, because the web server cannot save you from CGIs relative poor performance).
- How to run PERL scripts - #7 by Joe - Help! (Home for newbies) - Virtualmin Community
- Programming
- Simple CGI and Apache examples on Ubuntu Linux - Linux Tutorials - Learn Linux Configuration - Simple CGI and Apache examples on Ubuntu Linux
- Create first cgi script - YouTube | Greg Reichelt - Nice and easy to follow.
- CGI on C Programming - YouTube | beci67 - A multipart tutorial on making a simple C program and the deploying it to a CGI server. It starts of explaining the fundamental technologies.
- Python - CGI Programming - YouTube | Tutorialpoint - Very basic description.
- Python CGI Programming Tutorial | How to run CGI Programs in Python | Python Training | Edureka - YouTube - This Edureka video on 'CGI Programming In Python' will help you understand how we can write and Execute CGI scripts in Python.
- cgi — Common Gateway Interface support — Python 3.11.8 documentation
- depreceated in python
- CGI Programming with Perl
- Useful Examples
- FastCGI - Mythic Beasts
- A single script written in both CGI and FastCGI so you can see the differences.
- This is the most useful article I have found. It explains a lot.
- FastCGI — MapServer 8.0.1 documentation - An example of a FastCGI App and all of the relevant settings and procedures to get it running.
- PHP FastCGI Example | NGINX - How to configure NGINX with PHP FastCGI Process Manager.
- Remember FastCGI? - a moderately technical blog
- Alternative function for deprecated cgi - Python Help - Discussions on Python.org
- Simple CGI and Apache examples on Ubuntu Linux - Linux Tutorials - Learn Linux Configuration
- Simple CGI and Apache examples on Ubuntu Linux
- CGI works with multiple languages, but for now we start with bash shell.
- Examples for Bash, Perl, Python, C, C++
- FastCGI - Mythic Beasts
- Run CGI Apps on FastCGI
- fcgiwrap
- simple server to run CGI applications over FastCGI
- fcgiwrap is a simple server for running CGI applications over FastCGI.
- Its goal is to provide clean CGI support to the nginx webserver, although can be used with others.
- fcgiwrap is lightweight and has no configuration, making it possible to use the same pool to run different sites.
- How to execute CGI scripts using fcgiwrap | sleeplessbeastie's notes - I am an enthusiast of the uWSGI project. You are still not limited to it as you can use fcgiwrap, a very lightweight and straightforward FastCGI wrapper for CGI scripts that do not require additional configuration.
- GitHub - leejo/cgi-fast
- The new home for CGI::Fast, removing it from the original CGI.pm distribution - leejo/cgi-fast
- CGI::Fast is a subclass of the CGI object created by CGI.pm. It is specialized to work with the FCGI module, which greatly speeds up CGI scripts by turning them into persistently running server processes. Scripts that perform time-consuming initialization processes, such as loading large modules or opening persistent database connections, will see large performance improvements.
- FcgiWrap - Community Help Wiki
- fcgiwrap is a simple server for running CGI applications over FastCGI. It hopes to provide clean CGI support to Nginx (and other web servers that may need it).
- fcgiwrap can be used together with Nginx to serve CGI or Perl scripts (.cgi).
- How to execute CGI scripts using fcgiwrap | sleeplessbeastie's notes - I am an enthusiast of the uWSGI project. You are still not limited to it as you can use fcgiwrap, a very lightweight and straightforward FastCGI wrapper for CGI scripts that do not require additional configuration.
- fcgiwrap
- Alternatives
- ipc - Are there alternatives to CGI (and do I really need one)? - Stack Overflow
- Even under moderate loads, CGI is a pretty unscalable beast. FastCGI is an option, but you'll probably also find a mod_XXXX package where XXXX is the name of your language. There's a mod for ruby, perl, and python for instance and probably a fair few others.
- Is CGI still a thing? Which alternatives are there? | Reddit
- I stumbled upon CGI and tried to set it up on a server – provided by uberspace.de – but failed. Their support told me that CGI isn’t too relevant for them anymore, as basically any language has its own webserver nowadays and that I should just run the one I want as a service listening to a specified port
- CGI was a way of running a specific script through an http request.
- ipc - Are there alternatives to CGI (and do I really need one)? - Stack Overflow