• Home

Items filtered by date: December 2014

Saturday, 21 September 2024 15:08

Washed out colours on my Dell U2414H monitor

I have written these instruction while trying to diagnose and fix washed out colours on my Dell U2414H monitor attached to my Dell E6540 laptop which uses the Intel HD Graphics 4600 GPU however the solutions and logic will apply to al lot of setups.

Solutions

Some of these options might not realise unless you restart your PC.

  • Calibration (settings can be changed at the monitor and through Windows)
    • Run the Windows calibration tool (in particular, increasing Gamma)
      Use an Online Gamma Calibration Tool
  • Windows
    • check the brightness in the Action Center (bottom right of screen where screen snip is)
      • This will only affect the inbuilt panel
    • Make sure `Night light` is not enabled.
    • Make sure no 3rd party screen brightness utilities are running.
    • Disable accessibility colour filters
    • Make sure the monitor is set to native resolution and the correct refresh rate.
  • Intel / nVidia / AMD Utility
    • Ccreate a custom resolution in 1080p
  • Hardware
    • Use HDMI and put to full colour range
    • Use external USB graphics card
    • Use another graphics card (not so easy in a laptop)
    • Try different cables. The HDMI signal is especially susceptible to cheap cables.
    • Laptop display panels have a hardware brightness control, look at this, but should be tied to action centre brightness setting.
    • Cehc the monitor is being correctly detected.
  • Graphics Card
    • use the `Intel HD Graphics Control Panel` to alter the colour and gamma (or the nVidia/AMD equivalent)
    • nVidia cards only: This can be rectified for both DisplayPort and HDMI by creating a custom resolution with a 59.999Hz refresh rate. This will be treated by any application* as 60Hz but uses the correct Full Range RGB 0-255 colour signal. The process for setting this up is shown in the video below.
  • Dell U2414H monitor
    • Make sure it's device driver is installed and does not display as "Generic PnP Monitor" in device manage
    • Install the ICC profile for the Dell U2414H monitor. You will find this in the driver package.
    • Check the monitor's viewing angle. Depending on how you look at it the picture can look bad.
    • Check the monitors physical settings (colour, gamma, brightness, contrast, etc...). There is no harm in having a play as you can always put them back.
    • If you have several monitors of the same model, there can be differences between the revisions.
    • Your monitor is old and failing. Get a new one.
    • Try all of the ports (miniDP, DP, HDMI) and see if you get a better picture through one over the others.
    • Daisy chaining the DP ports should not cause any issue, but no harm i trying a single monitor on its own.
    • Try Display Port v1.2
    • Try your monitor with a known good laptop or PC to test to rule out the computer.
    • Confgure the monitor via `EDID` which I assume will require `DDC/CI` to be enabled. This monitor might not be EDID capable.
    • Enable DisplayPort 1.2
      • This might not be compatible with your devices and can cause the monitor to no longer recognise an input (not permanently though)
      • It might make the picture better.

Dell U2414H monitor settings

These are a reference for my monitors but if you have a Dell U2414H you might as well use them.

  • Custom Colour:
    • R:100,
    • G:98
    • B:990
  • Brightness: 100
  • Contrast: 75
  • DisplayPort: v1.2 (optional)

Research

  • Windows Calibration
  • Intel 4600 HD GPU / Intel  HD Graphics Control Panel Calibration
    • Best settings to fix washed out colors in Intel HD Graphics Control Panel? | Reddit
      • You need to set the quantization range to full range both under Display > General Settings or under Video > Color Enhancement > Input Range, where you have to click Use Driver Settings first.
      • HDMI tend to default to RGB limited mode, which is what non-HDR televisions use, but it's not suitable for computer displays.
    • Fix your laptop's color and brightness with Graphics Command Center | PCWorld - Fix Windows 10 auto-brightness and color problems with this hidden Intel app.
    • Potential fix for Intel graphics washed out colours (non native HDMI displays) | Reddit
      • When I tried to change the resolution of my monitor to something else I realised that my monitor was using the full colour range. It was only in the case of the default 1080p resolution that it would show washed out colours.
      • The solution was to create a custom resolution in 1080p
    • windows 10 - External monitor is washed out with Intel HD Graphics - Super User
      • This is what worked in my case. My monitor was being incorrectly detected as a HDTV with limited range and adding a custom resolution with "CVT-RB" timing fixed it.
      • How To Geek may have the answer! Essentially HDMI sometimes defaults to RGB Limited. You need to go into your graphics card control panel and change it to RGB Full. On Intel cards it's at the bottom of the General tab according to HTG's instructions.
      • The technical difference is that full range RGB uses full 8 bit range from integer value 0 to 255 where 0 is black and 255 is white. Limited range RGB transmits the image with 8 bit integers but all values 0-35 result in black and all values 235-255 result in white. As a result, the image is actually using only integer values 35-235 meaning you get better compability with old displays but lose about 22% of all possible colors due less effective bits available for the signal.
      • If GPU and display are both compatible, you always want full range RGB support.
      • Some technical information here.
    • How to Avoid Washed Out Colors When Using HDMI on Your PC - Black colors may look washed out and gray if you connect your PC to its display via an HDMI cable, and it's not your display's fault.
    • Quantization Range Option Is Not Visible in the Intel® Graphics
      • Describes expected behavior of the Quantization Range option that’s found on the Intel® Graphics Control Panel and the Intel® Graphics Command Center depending on video connection type.
      • With the most recent drivers (xx.xx.100.xxxx and newer), the Quantization Range option will only be visible if the video output on your computer is native HDMI. Using a DisplayPort (DP) video output or adapters such as DP-to-HDMI or USB Type-C to-HDMI will cause the option to disappear.
    • Quantization Range Option in the Intel® Graphics Control Panel or... - Expected behavior of the Quantization Range option that’s found on the Intel® Graphics Control Panel depending on video connection type.
  • 3rd Party Calibration add Tools
    • LCD monitor test images - Welcome to the Lagom LCD monitor test pages. With the test images on these pages, you can easily adjust the settings of your monitor to get the best possible picture quality.
    • Best free Monitor Calibration Tools for Windows 11/10 - Learn how to calibrate your monitor using these free Color or Monitor calibration tools for your Windows 11/10 PC. Good for Photographers or Gaming.
    • 10 Best Monitor Calibration Tools for Windows in 2022 - To calibrate your monitor, you’ll need calibration software, special calibration gadgets, or a combination of both. Let’s review software and hardware to find the ideal calibration tool for your monitor. 
    • Best Free Monitor Calibration Software Windows 10 - TechWiser - You can always buy a hardware colorimeter but, if you have zero dollars to spend, here is some free software that will get the job done. 
    • Gamma calibration - Lagom LCD test
      • This is a gamma calibration test image. The gamma defines how the luminance (the amount of light) on the screen depends on the 8-bit RGB values. As of 2007, computer monitors are supposed to adhere the sRGB standard, which is very similar to a gamma value of 2.2.
  • Screen Brightness Tools (ie night light)
  • Dell U241H Monitor
    • Dell U2414H Review | PC Monitors
      • A review of the Dell U2414H, a 23.8 inch AH-IPS member of the UltraSharp family with exceptionally thin bezels.
      • This can be rectified for both DisplayPort and HDMI by creating a custom resolution with a 59.999Hz refresh rate. This will be treated by any application* as 60Hz but uses the correct Full Range RGB 0-255 colour signal. The process for setting this up is shown in the video below.
    • Dell U2414H Review - TFTCentral - Dell's UltraSharp range always attracts quite a lot of attention and they always seem to do a good job of updating their screens to offer buyers something a bit new and different each time they do.
    • U2414H Banding using sRGB preset. | DELL Technologies
      • Q: My monitor is exhibiting banding with the sRGB preset. It doesn't do this on the Standard preset, but I would like to use the sRGB preset since that's the one that was factory calibrated. Is there anything I can do to correct this?
      • A:
        • Factory default does not over write calibrations. I would reset and check all modes.
        • Install the driver from the File Library. The driver includes the INF, ICM (ICC), CAT files.
    • Calibrating Dell U2410 and U2414H using Colormunki Display | DisplayCAL
      • Hi! I'm new to display calibration and bought and X-Rite Colormunki Display for calibration and profiling of my Dell U2410 and U2414h.
      • I followed an advice from some guys in the Dell forum and used the factory menu for OSD adjustment of the white point (because usually SRGB preset doesn’t offer RGB sliders).
  • Misc
    • If you turn all the brightness up full and a pure white page does not get any brighter then this is the maximum whiteness that the monitor will show white. If the white looks off colour at this point, then it is a limitation of the monitor.
    • sRGB - Wikipedia - sRGB is a standard RGB (red, green, blue) colour space that HP and Microsoft created cooperatively in 1996 to use on monitors, printers, and the World Wide Web.
    • DisplayPort - Wikipedia - DisplayPort (DP) is a proprietary[a] digital display interface developed by a consortium of PC and chip manufacturers and standardized by the Video Electronics Standards Association (VESA).
    • DISPLAYPORT 1.2, DISPLAYPORT 1.4, DISPLAYPORT 2.0 and DISPLAYPORT 2.1 - To help you understand the evolution and the differences between DP standards, today’s article will walk you through the differences between DisplayPort 1.2, DisplayPort 1.4, DisplayPort 2.0, and DisplayPort 2.1.
    • GitHub - dylanraga/win11hdr-srgb-to-gamma2.2-icm - Transform Windows 11's virtual SDR-in-HDR curve from piecewise sRGB to Gamma 2.2
Published in General
Read more...
Sunday, 15 September 2024 13:25

Transferring music from Sony MiniDiscs

There are a few ways to recover music from Sony Minidiscs and these will vary between devices:

  • Digital download
    • Digitally without loss.
    • This will be the actual audio file 'as-is' from the disk. (aea)
    • Tracks will be split into individual files.
  • Analogue
    • Output via the 3.5mm jack.
    • This is a universal method.
    • There will be no track data.
  • SPDIF / Optical / Coax
    • Digital with minimal loss
    • There will be no track data.

This article is dedicated towards digitally downloading tracks direct from the Walkman with no sound quality loss.

My Kit

  • Windows 11 Laptop
  • Edge browser (Google Chrome can be used)
  • Zadig
  • Sony Portable Minidisc Recorder MZ-N710 (NET MD Walkman)
  • USB cable (USB A to Mini-B)

My Instructions

This YouTube tutorial covers everything from grabbing tracks using Method 1 to converting the .aea files to a more useful format.

  • Connect device to the computer
    • Using the USB A to Mini-B cable.
  • Install Zadiag
    • Short instructions
      • Connect your MZ-N710 via the USB cable.
      • Download Zadig
      • Run the Zdiag installer
      • Click install
      • Reboot windows
    • GitHub - asivery/webminidisc
      • The Windows USB stack requires a driver to be installed to communicate with any USB device. The bad news is that there are no official Windows 10 drivers for NetMD devices. The good news is that we don't need it! We can just use a generic driver like WinUSB to access the device.
      • You can find installation instruction here, but the easiest way to install is to use Zadig.
  • In Edge
    • Open https://web.minidisc.wiki/
      • This is "Web MiniDisc Pro" and is a fork of the original but can also download tracks digitally.
    • Click connect
    • Select Netman MD
  • Safety
    • Make sure all your discs have write protect enabled to protect you from mistakes.
  • Insert your disc
    • Simply put your first disc into the MZ-N710
  • Download the tracks
    • Method 1 (use this)
      • Click on the 3 dots and select Enable Homebrew Mode Ripping in Main UI
      • Select all of the tracks you want
        • Clicking the top track, hold shit and then select the last track will select them all.
      • Click "Download"
        • We don't want to convert them here, we want the RAW data.
      • The track(s) will now download as .aea files labelled with their track number and name (if present).
    • Method 2 (old)
      • Enter Homebrew Mode
        • Click on the 3 dots and select Enter Homebrew Mode
      • Homebrew Mode explained
        • See: Homebrew mode [MiniDisc Wiki]
        • You will be now present with a some tables representing Tracks and the TOC
        • The "Track Junction Map" has one track per sector thing
        • Track 0 is not a track, it possibly is the TOC.
        • You can only download one track at a time.
        • Not all tracks are named
        • The unit must be power cycled between each disk (i think) to clear the TOC which also happens when you swap the disc.
        • If you press the Shift key the track names should be displayed but I never got this to work, maybe my tracks did not haven names on them.
      • Download the track
        • Click on a sector in "Track Junction Map"
        • Click on the download icon.
        • The track will now download as an .aea file labelled with their track number and name (if present).
  • Swapping Disc
    • Stop the disk spinning before ejecting
      • On my MZ-N710 if I hold the CANCEL/CHG button this will stop the disc spinning and allow you to swap the disc without restarting "Web MiniDisc Pro" but this does not always work.
    • You might need to restart "Web MiniDisc Pro".
    • Repeat the download method you choose.

Notes

  • General
  • Web MiniDisc / Web MiniDisc Pro
    • GitHub - asivery/webminidisc:
      • Upload your Music to NetMD and HiMD MiniDisc devices thanks to WebUSB and WASM
      • Differences between Web Minidisc and Web Minidisc Pro
      • Web MiniDisc Pro was forked from the original Web MiniDisc to provide a more advanced workflow for interacting with NetMD devices.
      • https://web.minidisc.wiki/is the new webapp. Web MiniDisc Pro = Brings NetMD Devices to the Web
    • Web MiniDisc Pro - Brings NetMD Devices to the Web
      • Bring back to life your old NetMD MiniDisc player. Upload music to MiniDisc from the Browser.
      • the new version of the webapp.
      • Web MiniDisc Pro - It is a complete replacement for Sony's SonicStage and Simple Burner software. It is considered the flagship NetMD software project.
    • Web MiniDisc - Brings NetMD Devices to the Web
      • Bring back to life your old NetMD MiniDisc player. Upload music to MiniDisc from the Browser.
      • This is the original version of the webapp.
    • The Web MiniDisc Application - Use your old NetMD device in the browser - Get control of your MiniDisc device thanks to WebUSB and WASM
      • Some of the last MiniDisc players, branded as NetMD units, were equipped with a USB port that allowed for recording music onto the device by using the infamously SonicStage software (Windows only, of course). When Sony abandoned the MiniDisc, that software was left unmaintained and, nowadays, it can’t run outside virtual machines or without using dangerously unsigned drivers.
      • I’ve been lucky enough to find my old MZ-N710
    • The Web MiniDisc Application - YouTube | Stefano Brilli - A short demo of how the Web MiniDisc application works.The GitHub project also contains a few instructions to make the app work fine if you're on Windows or Linux.
    • Web Minidisc Update: ToC Cloning and ATRAC download via USB on Type-S Sony NetMD portables | Reddit
      • I'm very happy to announce a new version of Web Minidisc Pro. It's now possible to transfer tracks via USB back to the computer from Sony Type-S portables.
      • To enable this new functionality, you must check the "Factory Mode Ripping In Main UI" switch in the ellipsis menu.
      • After enabling that functionality, the 'Record' button gets replaced by the 'Download' button. If a warning shows up, saying that the current version is not supported, please use the full factory mode to download the RAM and ROM, then contact me.
      • It can definitely rewrite the TOC of an errored disc, but if the disc itself is damaged, the data just won't persist, so it will go back to its old, errored self.
        • Has instructions on fixing a disc's TOC
      • Worked with my MZ-N710 today
      • You can convert them in vlc media player if you need to, under media /convert. Vlc also does batch conversion as well.
      • AEA files are supported by VLC you can convert them/batch convert them directly from vlc media player under Media/convert/save. (to mp3 should you want to).
      • It won't be possible with the MDS-S500. The button to enter the factory mode won't even be displayed. This works only on portables because of how they're built (the CPU and the USB controller are one chip, this is not the case in bookshelf systems / decks).
      • Sony made a tool to convert from ATRAC to MP3: https://www.sony.com/electronics/support/downloads/W0002971
    • Managing tracks [MiniDisc Wiki]
      • Web MiniDisc Pro has two ways to copy tracks on a MD back to the PC: recording and downloading. The Sony MZ-RH1 also allows downloads similar to the exploits method below.
      • If many options are disabled, the disc may be write-protected. Check the write-protect tab on the disc (open = writable; closed = protected).
    • Ignore vs Skip?
      • Bad Sector - Web Minidisc Pro | Reddit
        • asivery
          • Skip this sector discards this sector's data, and jumps to the next one.
          • Ignore writes the corrupted data to the output file you'll download.
        • NeoG_
          • Skip = Delete corrupted data , audio will jump slightly as part has been removed. Track length is slightly shortened.
          • Ignore = Keep corrupted data, audio will act unpredictably while going over the corrupted data such as making glitchy sounds, Track length is preserved.
    • Web Minidisc Pro - Atracdenc vs Remote Atrac encoder, which gives best quality ? | Reddit
      • This is for transferring music to the device
      • I also read this on the web minidisc pro guide page :
        • "SP mode does not need any encoder as the ATRAC processing is performed by the recorder. This also means SP mode records much slower than LP modes."
        • So I guess choosing the Remote Atrac Encoder, doesn't change anything quality wise, when you choose SP mode ?
      • The remote encoder uses Sony's propriety ATRAC3 encoder from the PSP SDK and has a much higher output quality when compared to atracdenc.
        • And indeed, the remote encoder does not impact SP (ATRAC1) audio.
  • Other Sites
  • Buying Guide
    • MiniDisc Buyer's Guide [MiniDisc Wiki] - MiniDisc Buyer's Guide This page is intended to be a compliment to the Getting Started guide, with more details about particular generations of machine, decks, and “gotchas” - for someone new to the MD format, please check the Getting Started guide first. What do you need?
    • All NetMD devices [MiniDisc Wiki] - All NetMD devices Page / USB Capable players
    • Sony MZ-RH1 / MZ-M200 [MiniDisc Wiki]
      • Sony MZ-RH1 / MZ-M200 The last portable MiniDisc recorder produced by Sony.
      • When originally released, it was the only device capable of transferring recordings made on standard MiniDiscs back to a computer. However, this functionality is now available to all USB-equipped HiMD portables as well as a number of NetMD portables thanks to the WebMiniDisc project. A list of the NetMD portables which support this feature can be found Here
    • Devices that support Homebrew features [MiniDisc Wiki]
      • This page lists Type-S and Type-R portables from Sony (as well as direct clones) that support the netmd-exploits as described on the Web Minidisc Pro guide.
      • The exploits for Type-S devices are older and more thoroughly tested. The exploits for Type-R devices currently operate slower and there may still be errors to work out, and not all functions are available.
      • As of this writing, ripping performance is roughly: - Type-R devices will rip at roughly half of real-time, for SP audio. - Type-S devices will rip at roughly 4x real-time, for SP audio.
  • Hardware
    • Write Protect
      • MINIDISC WRITE-PROTECT SWITCH - YouTube
      • Check the write-protect tab on the disc (open = writable; closed = protected)
        • Open mean the tab is there, closed the tab is pushed in so there is a gap/hole.
  • Sony MZ-N710
    • Buy a `Sony MZ-N710`
      • MZ-NF810 same as MZ-710 but with radio tuner.
      • The Italian programmer who originally wrote this software used his MZ-710.
    • Sony MZ-N710 [MiniDisc Wiki] - Sony MZ-N710 Compact and durable NetMD recorder, with its only significant downside being a reliance on gumsticks for internal power, not AA batteries. Works perfectly in both Web MiniDisc and SonicStage. Identical to the MZ-NF810, with the exception of lacking a TV tuner.
    • Sony RM-MC33EL [MiniDisc Wiki] - The remote.
  • Copying MiniDiscs to PC
    • Copying MiniDiscs to PC [MiniDisc Wiki]
      • This guide covers real time audio transfer from a MiniDisc to a PC for archival.
      • Be aware that the most common USB sound card with SPDIF input, the C-Media CM6206, does not accept inputs with a SCMS copy protection signal.
      • Optical and Coaxial digital signals can be converted with an inexpensive adapter.
  • Converting (ATTRAC / .aea) files
    • General
      • AEA files are supported by VLC you can convert them/batch convert them directly from vlc media player under Media/convert/save. (to mp3 should you want to).
    • File information
      • AEA File - What is a .aea file and how do I open it? - Learn about .AEA files and view a list of programs that open them.
        • An AEA file contains audio data encoded in the ATRAC1 format, which is a data compression format developed by Sony. It stores audio most likely dumped from a MiniDisc (MD), which is an optical disc similar to a compact disc (CD) that can be played with various MD players.
        • AEA files are not common, since MDs were introduced back in 1992 and overshadowed by CDs. You will most likely never encounter an AEA file unless you dump audio from an MD you own. You may also receive an AEA file from a friend sharing music dumped from one of his MDs.
        • The ATRAC1 format is a variation of the ATRAC format, which stands for Adaptive TRansform Acoustic Coding. Besides being used in MDs, the format was also utilized in the Sony Dynamic Digital Sound (SDDS) theater sound system in the 1990s.
    • CODEC information
      • ATRAC: Adaptive Transform Acoustic Coding for MiniDisc | minidisc.org
        • Compression of 16-bit 44.1 kHz stereo audio into less than 1/5 of the original data rate with minimal reduction in sound quality.
        • 16-bit 44.1 kHz = (16/44.1)
      • .AEA File Codec Info Question... | Reddit
        • DCT based audio formats don't really have a bit depth, so VLC decodes them all to 32-bit PCM internally. Newer versions of VLC identify this as "decoded bit depth" and report the original bit depth if available, but I suspect for atrac files it is not available, and only the sample rate is.
        • I'm still looking for the most viable way of converting ".aea" files extracted with WebMiniDisc Pro, and although ffmpg is a good option I have the doubt about wich are the best settings for the conversion to WAV or FLAC, since in the VLC info about the ".aea" file, it says that the bit depth is 32 bits wich I find too high but altough the source is only 16 bit I don't want to convert to 16 bit as I would be cutting information from the original minidisc file... so is trully 32 bit the depth of the minidisc files (SP)?
        • VLC worked fine for converting multiple at once but otherwise I used Audacity. For some reason VLC wouldn't play some of the tracks but Audacity does. You can use a software called SPEK to see if information is lost during conversion.
        • The AEA files should be 16/44.1.
        • The decoding is happening with the open source ATRAC1 decoder in ffmpeg/VLC and is itself passable but still not quite as good as if you'd played the audio on a minidisc machine and recorded it onto your computer.
        • I haven't tested converting the AEAs to anything yet but I'd say if the result is any greater than 320 kilobits per second, something is lying along the way. The SP format is only 292kbit/sec and ATRAC isn't that miraculous.
        • An alternative ATRAC decoder (and encoder...) exists in the form of "atracdenc" here: https://github.com/dcherednik/atracdenc
        • I'd just go with converting to a very common lossy format once now (320 kbps MP3 if you want to overdo it, but 128/192 is probably "just fine", and if it's spoken word only - not music - even less than that should be plenty - YMMV) for everyday use and easy accessibility, and keep the AEA files for long-time archival (you can always turn the AEA files into WAV/FLAC using ffmpeg).
        • ffmpeg's source code won't go away that fast, so there will always be the possibility of converting later (assuming the AEA files are smaller than a corresponding FLAC file, so you can save storage space by hanging onto the AEA files + ffmpeg instead of just FLAC). Also, VLC will most likely "forever" be able to play back AEA files on the fly.
    • Guides
    • Tools
    • Other Audio Containers
  • Troubleshooting

 

Published in Other Devices
Read more...
Thursday, 12 September 2024 08:53

Divi Contact form will not send if using a free email address

Background

This article is based on the server running cPanel/WHM, the website CMS is WordPress with Divi as it's template, but the logic will most likely apply to a lot of other platforms.

Issue

I have a contact form that will not send the emails when the submitted details contain a free email address even though the contact form says the message was sent.

The failed emails do not appear in "Track Delivery" in cPanel so I have nothing to inspect and figure out what is going on.

If you do not manage your emails locally then an emails that are sent but are bounced will be returned to the server defined in the MX entry.

Cause

  1. Outbound emails are being scanned for SPAM by SpamAssassan (spamd).
  2. The email in the `Email Address` field on a Divi contact form is used by default for the <reply-to> header.
  3. Emails with freemail addresses present in the header score extra SPAM points.
  4. Because of the extra points caused by having freemail addresses present, the emails are scored just above the SPAM score threshold and they become flagged as SPAM.
  5. Flagged emails are modified and then delivered to the system mailbox in cPanel, they are never passed to the MTA (Exim).
    • The modification and redirects are performed by spamd.
    • cPanel "Track Delivery" shows transactions by Exim, but because the email never gets to Exim, there will be no record of the delivery or transport of the email.
    • A modified email in the system mailbox might look something similar to this:
      ### Title ###

Mail failure - rejected by local scanning code

### Message Body ###

A message that you sent was rejected by the local scanning code that
checks incoming messages on this system. The following error was given:

  This message was classified as SPAM and may not be delivered

------ This is a copy of your message, including all the headers. ------

.....

Solutions

These are some solutions an workarounds, pick which ever best suits your needs, which ever you pick you should monitor the situation for a while to make sure the changes you implement are working as expected.

Use SMTP plugin for WordPress

  • Using this method reduces the outbound SPAM score massively
  • Requires you to setup an email account for every website. Do not use the cPanel system account for emails as this is a security risk.
  • Your server does not need to handle email for the domain for you to be able to create an email account locally and send from it as normal, however I do recommend you make sure the SPF record is correctly set and that DKIM is not going to be an issue.
  • My recommended plugin is: FluentSMTP | WordPress.org

Remove the Freemail address from the email header

  • When using Divi, the email address on a contact form is used by default as the reply address Moving the client's email address to the content only would most likely reduce the SPAM score below 5 and therefore all the email to be sent.
  • In Divi you would use Message Patterns to alter the content of the email.

Increase the outbound SPAM score threshold

  • This change would be global
  • WHM --> Exim Configuration Manager --> Basic Editor --> Apache SpamAssassin Options -->
    • Scan outgoing messages for spam and reject based on the Apache SpamAssassin™ internal spam_score setting: Off
    • Scan outgoing messages for spam and reject based on defined Apache SpamAssassin™ score: 8.5
    • Do not forward mail to external recipients if it matches the Apache SpamAssassin™ internal spam_score setting: Off
    • Do not forward mail to external recipients based on the defined Apache SpamAssassin™ score: 8.5

NB: 8.5 = Default Threshold + PHP_SCRIPT + KAM_COUK

Disable outbound SPAM scanning

  • This change would be global.
  • This is not the preferred method as there is not outbound SPAM checking which can allow abuse.
  • WHM --> Exim Configuration Manager --> Basic Editor --> Apache SpamAssassin Options -->
    • Scan outgoing messages for spam and reject based on the Apache SpamAssassin™ internal spam_score setting: Off
    • Scan outgoing messages for spam and reject based on defined Apache SpamAssassin™ score: Disabled
    • Do not forward mail to external recipients if it matches the Apache SpamAssassin™ internal spam_score setting: Off
    • Do not forward mail to external recipients based on the defined Apache SpamAssassin™ score: Disabled

Disable certain SpamAssassin tests

You can reduce the SPAM score by disabling or cancelling out certain tests, but this requires a lot more time to setup.

Notes

WordPress - Email subsystem

  • mail()
  • Email Logging
  • Code Walk through 
    ## wordpress/wp-includes/PHPMailer/PHPMailer.php

'instantiate' => 'Could not instantiate mail function.',

-->

mailPassthru($to, $subject, $body, $header, $params)

-->

$result = @mail($to, $subject, $body, $header, $params);

-->

mail()              # This is a wrapper for sendmail

-->

sendmail            # cpanel sendmail is blocking gmail addresses

-->

sendmail returns true or false

WHM - Settings

  • General
  • Tutorial
  • SpamAssassin’s internal spam_score
    • Is calculated by summing the scores of various matched rules.
    • This is not the SPAM threshold, but just the score that SpamAssassin assigns to the email after all of the tests.
    • How to Configure SpamAssassin in cPanel – PhilmoreHost
      • Configuring your internal SpamAssassin score threshold = Specify a score under Apache SpamAssassin bounce spam score threshold.  ?
  • cPanel Tutorial Series 6: using email - TransIP - This is the sixth part of our cPanel Tutorial Series.
    • Apache SpamAssassin ™ reject spam score threshold
      • Incoming messages that are designated as spam are stopped instead of subjects being rewritten (see Filters), or the mail is moved to your spam box. You will never see messages that meet these criteria in your mail. If you use this option, we advise you to use a fairly generous score so only messages that are definitely spam are stopped, for example with a score of 10-11.
    • Apache SpamAssassin ™ bounce spam score threshold
      • Sends a bounce email to the sender of spam when mail reaches the set score. This can work positively and negatively: a conscious spammer knows that you are using a filter and will try to find something to work around it, but someone with an infected computer / server will find out more quickly that he or she has an infected computer / server thanks to a bounce message.
    • Scan outgoing messages for spam and reject based on the Apache SpamAssassin ™ internal spam_score setting
      • Scans all outgoing messages from your VPS and stops all messages that reach the set spam score (default 5). 
        This is a very useful option that we highly recommend. For example, suppose you created cPanel accounts for your customers and they host their own sites but never update them, then there is a plausible chance that sooner or later a vulnerability will be found, and a domain will start sending spam. Thanks to this option, you get rid of that spam, so your VPS does not get a bad IP reputation.
    • Scan outgoing messages for spam and reject based on defined Apache SpamAssassin ™ score
      • This option does the same as the previous one, but this allows you to determine the spam score upon which SpamAssassin takes action when outgoing mail is recognized as spam.
    • Do not forward mail to external recipients if it matches the Apache SpamAssassin ™ internal spam_score setting
      • Mail that is addressed to your VPS and which is redirected is also scanned by SpamAssassin and stopped if it is marked as spam.
    • Do not forward mail to external recipients based on the defined Apache SpamAssassin ™ score
      • Same as the previous option but allows you to set a score yourself.
    • Score
      • By default, SpamAssassin marks mail as spam when it reaches a score of 5. This base value can only be adjusted via command line using the following commands: 
        sudo nano /etc/mail/spamassassin/local.cf
      • The particular rule to alter is: 
        #   Set the threshold at which a message is considered spam (default: 5.0)
#
# required_score 5.0
  • acl_not_smtp and acl_smtp_data
    • Both acl_not_smtp and acl_smtp_data can count towards the Apache SpamAssassin™ internal spam_score setting, depending on how your Exim configuration is set up.
    • There is no proper documentation from cPanel for these options.
    • Ypu can expose the underlying code in the GUI by using the 'Advanced Editor'.
    • acl_not_smtp:
      • outgoing_spam_scan (Scan outgoing messages for spam and reject based on the Apache SpamAssassin™ internal spam_score setting)
      • This refers to Exim's Access Control Lists (ACLs) that apply to non-SMTP email submissions, such as mail sent from local scripts or PHP applications. This is often used to set rules for handling outbound mail generated by websites, contact forms, or local applications.
      • Purpose: To control and filter outgoing email messages that are not sent via SMTP, such as web apps or local daemons.
      • Advanced Settings (Under Advanced editor):
        warn
        condition   = ${if forany{<, $recipients}{!match_domain{${domain:$item}}{:+relay_domains}}}
        set acl_m_outbound_recipient = 1

warn
        condition   = $acl_m_outbound_recipient
        condition   = ${if <={$message_size}{1000K}}
        condition   = ${if !eq{$originator_uid}{0}}
        condition   = ${perl{spamd_is_available}}
        set acl_m_spam_scan_enabled  = 1

deny
        condition   = $acl_m_outbound_recipient
        condition   = $acl_m_spam_scan_enabled
        spam        = cpaneleximscanner/defer_ok
        message     = This message was classified as SPAM and may not be delivered
        log_message = "SpamAssassin as cpaneleximscanner detected OUTGOING not smtp message as spam ($spam_score)"

warn
        condition   = $acl_m_outbound_recipient
        condition   = $acl_m_spam_scan_enabled
        log_message = "S
    • acl_smtp_data:
      • outgoing_spam_scan (Scan outgoing messages for spam and reject based on the Apache SpamAssassin™ internal spam_score setting)
      • Purpose: The acl_smtp_data ACL is used to inspect the content of the email before it's accepted for delivery. It is commonly employed for tasks such as:
        • Spam Filtering: Checking the email content against spam filters (like SpamAssassin).
        • Virus Scanning: Scanning the email for malware or viruses.
        • Attachment Filtering: Restricting certain types of file attachments or scanning them.
        • Rate Limiting or Message Size Checking: Ensuring that message sizes adhere to server limits.
        • Custom Message Rejection: Applying custom rules to reject messages that contain specific content or violate policies.
      • Advanced Settings (Under Advanced editor):
        warn
        condition = $acl_m_outbound_recipient
        condition = ${if <={$message_size}{1000K}}
        condition = ${if !eq{$acl_c_authenticated_local_user}{root}}
        condition = ${if !match{$authenticated_id}{\N^__cpanel__service__auth__[^+%:@]+$\N}}
        condition = ${perl{spamd_is_available}}
        set acl_m_spam_scan_enabled = 1


deny
        condition   = $acl_m_outbound_recipient
        condition   = $acl_m_spam_scan_enabled
        spam        = ${if eq{$acl_m1}{}{cpaneleximscanner}{$acl_m1}}/defer_ok
        message     = This message was classified as SPAM and may not be delivered
        log_message = "SpamAssassin as ${if eq{$acl_m1}{}{cpaneleximscanner}{$acl_m1}} detected OUTGOING smtp message as spam ($spam_score)"

warn
        condition   = $acl_m_outbound_recipient
        condition   = $acl_m_spam_scan_enabled
        log_message = "SpamAssassin as ${if eq{$acl_m1}{}{cpaneleximscanner}{$acl_m1}} detected OUTGOING smtp message as NOT spam ($spam_score)"

 

WHM - Outbound Filtering

  • I am not sure if it uses the per user/cPanel custom rules (as appropriate) for calculating the spam_score when outbound filtering, I cant see why it doesn't though.
  • When an email has been forwarded after being scanned for SPAM it will have an additional header with it's score.
  • How to configure Spam Assassin to scan outgoing mail. – cPanel - This article covers configuring outgoing mail to be scanned by Spam Assassin.
  • Enable Outgoing Spam Filtering Using SpamAssassin - eukhost - This guide explains how to enable Outgoing Spam Filtering Using SpamAssassin.
  • Configure SpamAssassin to block outgoing form mail – cPanel
    • There are actually two assigned spam scores, one when you receive the email locally and one that is assigned to it when it's scanned outbound.
    • In this case Spam Score: X-Spam-Score: 100 refers to the score assigned on delivery to the server.
    • The score that is being assigned to the server when it is sent is the Outgoing Spam Score which in this case is below the threshold of 2: X-Outgoing-Spam-Status: No, score=1.7 So with a spam score of 1.7 SpamAssassin isn't seeing this email as spam and sends it.
  • How-To Prevent Exim From Forwarding Spam – cPanel - This article covers configuring Exim to not forward mail based on the message's Apache SpamAssassin™ Spam score.

SpamAssassin - Example rules that can be triggered

To find the rules, search for them in the following format: describe KAM_DMARC_STATUS

SpamAssassin - Managing rules

  • General
  • KAM Ruleset
    • This is not part of the Spamassassin base code.
    • The is a 3rd party ruleset that is included with cPanel/WHM and is on by default.
    • In cPanel/WHM: /etc/mail/spamassassin/KAM.cf
    • KAM Ruleset | The McGrail Foundation
      • The KAM Ruleset is a set of Apache SpamAssassin rules developed and used on our systems.
      • This ruleset has been in active use and development since May 2004 and provides a significant boost to the performance and efficacy of a stock installation of Apache SpamAssassin.
    • KAM Ruleset Channel | The McGrail Foundation - The McGrail Foundation Home Page

SpamAssassin - What rules caused an email to get flagged?

  •  Exim Mail Log - Example Entry 
    Sep  1 13:56:08 srv spamd[414141]: spamd: connection from localhost [127.0.0.1]:43490 to port 783, fd 6
Sep  1 13:56:08 srv spamd[414141]: spamd: setuid to cpaneleximscanner succeeded
Sep  1 13:56:08 srv spamd[414141]: generic: trusted_networks doesn't contain internal_networks entry '0/0'
Sep  1 13:56:08 srv spamd[414141]: spamd: checking message <XXXXXXXXXXXXXXXXZldVRrQX6dyBuRHk9yR9jnJNGRM@www.example.co.uk> for cpaneleximscanner:992
Sep  1 13:56:18 srv spamd[414141]: spamd: identified spam (5.9/5.0) for cpaneleximscanner:992 in 10.3 seconds, 3794 bytes.
Sep  1 13:56:18 srv spamd[414141]: spamd: result: Y 5 - FREEMAIL_FORGED_REPLYTO,KAM_COUK,KAM_DMARC_STATUS,PHP_SCRIPT,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED scantime=10.3,size=3794,user=cpaneleximscanner,uid=992,required_score=5.0,rhost=localhost,raddr=127.0.0.1,rport=43490,mid=<XXXXXXXXXXXXXXXXZldVRrQX6dyBuRHk9yR9jnJNGRM@www.example.co.uk>,autolearn=no autolearn_force=no,shortcircuit=no
  • spam - Where can i find explanation of a SpamAssasin scores like SPOOFED_FREEMAIL? - Server Fault 
    ## This adds the score
/var/lib/spamassassin/3.004006/updates_spamassassin_org/20_freemail.cf

## This is the list of the Freemail domains
/var/lib/spamassassin/3.004006/updates_spamassassin_org/20_freemail_domains.cf

## An old list
/var/lib/spamassassin/3.004006/updates_spamassassin_org/20_freemail_mailcom_domains.cf
  • How to find the descriptions of SpamAssassin rules to help understand why a message was marked as spam – cPanel - If you find that SpamAssassin is marking messages as Spam when it should not be, or that SpamAssassin is not marking messages as spam when it should be, you can use the following procedure to find out what rules were applied to a message and review the description of the applied rules for additional context.
  • How can I check why SpamAssassin applied a particular score? – cPanel - Some email messages are flagged or rejected as spam, but I'm not sure why. Can I check how SpamAssassin is applying this score?
  • How to find SpamAssassin scan results – cPanel -When SpamAssassin scans an email the results are saved to the /var/log/maillog file. This can be used to determine what rules are being triggered by the message. 
    /usr/local/cpanel/logs/spamd_error_log

SpamAssassin - Here’s a breakdown of how this score is calculated:

I got this from my VPS provider:

SpamAssassin’s internal spam score is calculated by summing the scores of various matched rules.

  1. Rule-Based System
    • SpamAssassin uses a large set of predefined rules to analyze various parts of an email, including:
      • Email headers (e.g., suspicious sender domains, mismatched or forged headers).
      • Message body content (e.g., words commonly associated with spam, HTML content, or embedded links).
      • Attachments (e.g., dangerous file types, large attachment sizes).
      • Blacklists (e.g., checking if the sender's IP address or domain is on a spam blacklist).
    • Each rule is assigned a specific score, positive or negative, depending on how indicative it is of spam. These scores are predefined, but you can customize them in SpamAssassin’s configuration files.
      • Positive score: Increases the likelihood that the message is spam.
      • Negative score: Decreases the likelihood that the message is spam (e.g., messages from trusted sources might get a negative score).
  2. Cumulative Scoring
    • When SpamAssassin scans an email, it applies multiple rules. The total spam score is the sum of all the individual rule scores that match the email.
    • Example: 
      Rule    Description    Score
HTML_MESSAGE    Email contains HTML    1.0
BAYES_99    Bayesian filter suggests 99% spam probability    3.5
RCVD_IN_SPAMHAUS    Sender's IP is in the Spamhaus blacklist    2.0
DKIM_VALID    Email has a valid DKIM signature    -1.0
Total Score        5.5
      • In this example, the email’s total spam score is 5.5.
  3. Spam Threshold
    • The total score is compared to the spam threshold configured in cPanel/WHM. The default threshold is usually 5.0. If the total score exceeds this threshold, the message is flagged as spam.
      • Spam Threshold of 5.0: If an email’s score exceeds 5.0, it is considered spam.
      • Spam Threshold of 7.0: If the threshold is set to 7.0, only messages with a spam score higher than 7.0 will be flagged as spam.
    • You can adjust the spam threshold score in the SpamAssassin configuration settings to make the filter more or less aggressive.
  4. Bayesian Filtering (Adaptive Learning)
    • SpamAssassin also uses a Bayesian filter, which is a machine-learning technique that adapts to the types of emails received over time. When trained with both spam and non-spam emails, this filter becomes more accurate in determining whether an email is spam by calculating the probability based on previously learned patterns.
      • The Bayesian filter assigns probabilities to words or phrases, which contribute to the overall spam score.
      • SpamAssassin rules like BAYES_50 (50% spam probability) or BAYES_99 (99% spam probability) are examples of how this filter impacts the overall score.
  5. Custom Rules and Scoring
    • You can also create custom rules and assign them specific scores. This allows you to adapt SpamAssassin to your organization’s needs by:
      • Increasing the score of certain patterns (e.g., emails with specific attachments).
      • Decreasing the score for emails from trusted domains or senders.
  6. Adjusting the Spam Score Multiplier
    • In cPanel and WHM, the spam score is multiplied by 10 for internal use. For example, if you set a spam score threshold of 5.0 in WHM, it will show up as 50 in log files and email headers.
    • For instance, if you see the following in an email header, It corresponds to a score of 4.0 (since 40 divided by 10 is 4.0), which is below the default spam threshold of 5.0, so the email wouldn’t be flagged as spam.
      X-Spam-Score: 40

 

Published in General
Read more...
Thursday, 01 August 2024 10:23

My Meta Quest Notes

These are a collection of my notes for the Meta Quest 3

General

  • Official Sites
  • Other Sites
  • Specs
  • Performance
  • Equipment
    • VR Treadmills
    • Controller Stocks
  • Maintenance / Looking after your Quest
  • Legal / Warranty
    • Meta might be trying to get around EU 2 year warranty with Quest 3 or they already did | Reddit
      • Consider Duration from the T&Cs: 1 year from the date of purchase or delivery of the Product, whichever is later (the “Warranty Period”). If you are a consumer, this does not affect your statutory rights which may provide for longer warranty coverage.
      • You're mixing the legal guarantee (2 years in the EU, regarding the seller of the product) with the commercial warranty (1 year, in most cases regarding the manufacturer). A lot of companies only offer 1 year of commerical warranty (like Apple for example).  
      • this. also important to know: the one year warranty that apple and meta offer are voluntary. but as nightkrwlr said: it's different from the two-year consumer law rule.
      • = 2 Years warranty
    • Meta Quest EU/EEA/UK consumer right of withdrawal information | Meta Quest - Under applicable law, consumers in the European Union (EU), the European Economic Area (EEA) and the UK have a right to withdraw from or cancel certain purchases for a refund.
    • Meta Horizon Store Terms | Meta Quest - No meta description
  • Sign up issues
    • The sign up form will only accept UK postcodes in this format SW1 1LF and not SW11LF
  • Meta keep trying to charge £59.99 for my free subscription
    • I could not get the 6 months free to work, my bank kept on wanting to charge me £59.99 even thought it said the due amount today was £0.00
      By ticking this box and clicking the Confirm button, you agree that the content will immediately begin downloading onto your device, and you acknowledge that you will thereby lose any statutory right of withdrawal (EU/EEA) or cancellation (UK). Learn more
You will be charged £59.99, including tax, when this free trial ends on 26 Feb 2025 and every year when this subscription is automatically renewed until you cancel. Cancel at any time in your account settings at least 24 hours before the end of this free trial or the next billing date to avoid future charges.By selecting Confirm, you agree to the Meta Quest Store Terms.
    • So this is what I did to fix it:
      • Created a virtual card with £10 on it to prevent my account from getting charged.
      • Using the phone APP I signed up using my information and the virtual card
      • When I get to the point it wants to test my card is valid, the amount being charged to my account is £59.99
      • A 2FA notice appeared in a browser thing, I clicked cancel on it ignoring any messages on my phone to accept or deny the transaction through my banking App.
        • This is not the link that is sent to your phone's banking app asking you to approve.
        • Never approve it and don't cancel it either.
        • I let the request in my banking app expire. It should only take 10 mins.
      • Bingo, it works
    • Research
  • Cloud Backup
    • About cloud backup | Meta Store
      • When cloud backup is turned on, Meta backs up your device's app data, such as app progress or app settings, to the cloud. This allows you to easily pick up where you left off in a game if you reinstall an app, factory reset your device or set up a new device.
      • To keep your cloud backup data safe, your app data is encrypted on the device and remains encrypted on our servers.

Meta Quest+

  • Meta Quest+ subscription | Meta Store
    • Experience the ultimate gaming adventure with Meta Quest+. Enjoy instant access to over 25 games, exclusive deals and monthly drops. Cancel at any time with no hidden fees.
    • Meta Quest+ is your subscription for premium games and more. Discover and play on repeat. You'll get instant access to a growing content catalogue and curated monthly drops of top titles, all for just £7.99 a month. Plus, you'll get exclusive deals and offers as a subscriber. Get ready to explore new apps and worlds.
  • Meta Quest Plus - Homepage | Meta Store
    • Claim your free games here every month, both the permanent ones and the ones in the rotating games catalogue.
  • You get a selection of free games every month in 2 categories, both of which require you to redeem them to gain access but they have different retention terms.
    • Monthly Games
      • Every month you are given 2 games you can redeem.
      • You need to redeem these games, when offered, to keep them.
      • Once redeemed, these games are added to your library permanently.
    • Games catalogue
      • The majority of the game catalogue will remain consistent month over month, but each month, one or more titles may leave or be added in order to keep content fresh.
      • if you have a redeemed a game that is removed from the `Games catalogue` you will loose access to it and will need to purchase it to continue using it. Your game saves should remain preserved.
  • What happens if I unsubscribe or resubscribe?
    • Once you are no longer a paying Meta Quest+ subscriber, you will lose access to all previously redeemed Meta Quest+ titles and not be able to redeem any more titles.
    • When you resubscribe, you will regain access to all previously redeemed Meta Quest+ titles and the titles that are currently available for as long as you are a subscriber.
  • Examples of Meta Quest+ game listings
    • Redeem
    • Redeemed
    • Redeemed - Soon to be removed from the `Games catalogue`

 

Q&A

Apps

  • System
  • YouTubeVR
    • Experience your favourite YouTube channels, videos and creators in virtual reality.
    • The YouTube VR app turns every video on the platform into your own virtual reality experience and reimagines YouTube as a 3D world you can explore from the inside.
    • Cannot load the app, keeps on crapping out
      • 5 Ways to Fix YouTube VR if It’s Not Working on Oculus Quest - To fix Oculus Quest YouTube VR not working issue, you will need to tweak some settings and follow additional steps shown in this article.
      • Solution
        • Setup you boundary area before you start YouTubeVR
        • You must share boundary information when asked otherwise the app will just crap out with no message.

Casting and Inputting

Remote Desktop / Productivity

Casting from Quest to (Phone|Computer)

Play PCVR Games on Quest

HDMI Input (Xbox One / SteamDeck / Playstation)

Xbox One

Play Xbox games on Quest (General)

You can pair your Xbox controller on your Quest for using with games.

  • Methods
    • Xbox One HDMI Output --> HDMI USB capture dongle --> quest
    • Xbox Gamepass
      • is natively supported on Quest 3
      • A.k.a Xbox Cloud Gaming (beta) / Xbox App
      • No Xbox needed
    • Xbox --> PC --> Quest
      • Uses Virtual Desktop App on the Quest and PC.
    • As PC VR Headset (needs a decent PC)
  • 4 Ways to Play Xbox Games On Oculus Quest / Quest 2 – Smart Glasses Hub
    • Can you also use the Oculus Quest and Quest 2 VR headsets to enjoy some Xbox gaming?
    • The short answer is that you can play Xbox games on both the Oculus Quest and Quest 2 headsets.
    • It is possible to stream the Xbox console to your headset or alternatively play Xbox games without the console by using the Xbox Cloud gaming service.

Play Xbox games on Quest (via Xbox App)

  • Playing Xbox on Oculus Quest 3 | OculusQuest | Reddit
    • I got the Xbox app running on the Oculus Quest 3. For anyone wanting to know how it's done, follow the steps below.
      1. Enable developer mode on Oculus. The requires a developer account which anyone can set up. It's super easy. This give you access to the library folder for unknown sources, more on this later.
      2. Install Side Quest on a PC
      3. Download File Manager APK from https://flashlight-clock-file-manager.en.uptodown.com/android
      4. Follow Side Quest steps to connect your Oculus
      5. Upload file manager to Oculus via side quest
      6. Get on your Oculus and use the web browser to search for the Xbox app APK and download from uptown.
      7. Go to your library, click the search bar, use the drop down to go to "unknown sources"
      8. use the file manager to install the Xbox APK
      9. connect an Xbox controller to the headset via Bluetooth setting
      10. log into Microsoft account and enjoy!
      11. It seems like a lot of steps but it only took me about 20 minutes altogether.
    • Other Comments
      • For some reason the xbox app uses low resolution (I'm using quest pro), still playable but sometimes annoying
      • Surprisingly low in my experience, but it looks like they use a low bitrate to achieve that so it doesn’t look great. There’s a custom Xbox remote play app someone made that lets you adjust the bitrate so that may help.
      • I know Xbox Cloud Gaming is officially coming to the Quest in December and I've seen some people assume that means you'd be able to remote play as well, but I'm not actually sure that'll be the case since I don't think they've specifically mentioned remote play.
      • For those wondering about the screen resize, open this menu on the oculus and hit "switch view". After that you should be able to resize windows.
      • Also, if anyone has any issues where the Xbox controller uses head tracking, press the start and back button on the controller at the same time and it will switch the controls.
      • Sidequest (advanced) can install apks directly to the oculus headset
      • FYI, there is a way easier way to do this. Just download Fluid on App Lab. Works out of the box.
  • How to play Xbox games on the Meta Quest 3 | Trusted Reviews - You can stream Xbox games directly to your Meta Quest 3, as well as the Quest 2 and the Quest Pro. Here's how.

Casting to a Xbox One

Casting from a Xbox One

  • How to Stream Xbox One to Your PC
    • You can stream Xbox One to any PC as long as they're both running Windows and are connected to the same network. Here's how to do it.
    • Page is faulty, click on the TOC links on the left hand side to navigate.

Play Xbox games on your PC

Purchases

random thing to look at later, I have not made a descicion and do not know the quality of the brands, this is more an idea holders

Troubleshooting

  • Solved: Meta Quest app - stuck on the "Pair a new device" ... - Meta Community Forums - 1153444
    • Q:
      • I'm stuck on the "Pair your headset with the Meta Quest app to continue." step during the Meta Quest 3 setup. The smartphone app tells me to "Allow permissions", which I have already done. I have "Use precise location" enabled as well. Pressing the blue "Continue" button does nothing. The Meta Quest 3 button is greyed out. I have a code displayed in the headset that I'm supposed to input somewhere in this app. What do I do?
      • My Meta Quest app is up to date. I have already installed all system updates on my smartphone. My smartphone and Meta Quest 3 are on the same WiFi network.
    • A:

      • I had the same problem. Make sure your location is turned on from the swipe down panel.
  • Solved: Passthrough turns on - can't use menu button - Meta Community Forums - 1078491
    • Q: My Oculus would not load the menu screen without passthrough on. With passthrough on the oculus would not load apps/games. The app on my phone says the apps are open on the device. 
    • A:
      Hey there! Thank you for letting us know the experience you're having with your device as this is not what we want. We do have a few users having the menu button disappear and a Factory Reset has gotten them back on track. However, I am not seeing some users not having the button load overall even when factory resetting. 
  • Stuck on Passthrough with Menus not working - Meta Community Forums - 1173053
    • Q: I'm stuck in pass through but the home music cuts in and out and I can't open any menus or nothing
    • A: With the device being stuck viewing your play area, we'd like to hard reboot the headset by holding the power button down for 30 seconds. 
  • Hand tracking not working
  • Cannot login to the Meta Quest community forum

    This call accesses the User ID platform feature, which requires certifying through the Oculus Data Use Checkup program. Please visit this page for more detail: https://developer.oculus.com/distribute/publish-data-use/. If you have already completed a Data Use Checkup, please ensure that all compliance issues are resolved by visiting the Compliance Dashboard: https://developer.oculus.com/resources/publish-compliance-dashboard/.
    • If you get this error below when you try and login to the Meta Quest community forum then you should know:
      • this is nothing you have done but is the garbage infrastructure and support team from Quest. A billion dollar corporation cannot keep their own API key updated and then when it needs swapping takes nearly 2 weeks to fix. Meta need to change their support team from wherever they have outsourced it to.
      • This is a common occurrence
    • Solved: Login to user forum problems | Meta Community Forum
    • Developer account data use checkup. | Overclockers UK Forums - Has anyone with a dev account had this pop up when trying to log into the meta forums
  •  not authorized to relay messages to the server that reported this error
  • Meta Quest app - stuck on the "Pair a new device" screen - Meta Community Forums - 1153444
    • Q:
      • I'm stuck on the "Pair your headset with the Meta Quest app to continue." step during the Meta Quest 3 setup.
      • The smartphone app tells me to "Allow permissions", which I have already done.
      • I have "Use precise location" enabled as well.
      • Pressing the blue "Continue" button does nothing.
      • The Meta Quest 3 button is greyed out.
      • I have a code displayed in the headset that I'm supposed to input somewhere in this app.
      • What do I do?
    • A:
      • Make sure your location is turned on from the swipe down panel.

 

Published in Other Devices
Read more...
Tuesday, 16 July 2024 12:02

My Pond Notes

These are my notes on rebuilding my pond

Purchasing

Tutorials

 

 

Published in House
Read more...
Saturday, 04 May 2024 08:11

My Discourse Notes

Just a collection of my notes for installing and using Discourse

  • General Integration / SSO
  • WordPress Intergration
    • How to Use Discourse Community Forums with WordPress - Discourse is one of the most popular community forums software platforms available today. Unlike a lot of its WordPress-based competitors, Discourse doesn’t run off of WordPress architecture. It has to be contained on its own server. That shouldn’t scare you, though. The developers understand that 1/3 of the internet can’t be ignored (WordPress’ market share), so they made it exceptionally easy to make these two powerhouse platforms play nice with each other. With the official Discourse WordPress plugin, you can make these two platforms interact like they were made with the other in mind.
    • Automatically Sync ALL WordPress users to Discourse - wordpress - Discourse Meta - I need all existing Wordpress users Synchronised/Imported into Discourse so that they are active Discourse users WITHOUT the users needing to first login to Wordpress to trigger the SSO Discourse account creation process. Is this a feature that’s currently available or do we need to develop a custom feature for this?
    • How To Integrate Discourse Forum With WordPress: 2024 Guide - Learn how to integrate your Discourse Forum with WordPress blog in this tutorial - Installing Discourse WordPress Plugin and configuring the settings. The main highlights include the ability to publish selected posts as Forum topics on Discourse and use Discourse for comments and discussions for WordPress posts.
  • Joomla Integration
    • How to Enable Single Sign-on (SSO) in Joomla - Joomlashack
      • Single sign-on (SSO) is a very useful tool for managing multiple sites.
      • Single sign-on is often used by large companies that have many different software platforms.
      • If all those platforms are connected using Single sign-on, then users only need to remember one username and password.
    • SAML SSO for Joomla, by miniOrange - Joomla Extension Directory - Joomla SAML Single Sign-On (SSO) allows you to secure your Joomla site by allowing users to login into Joomla site by different SAML Identity Providers like Azure AD, Okta, GSuite / Google Apps, Keycloak, ADFS, Salesforce, Office 365, Shibboleth2, Shibboleth3, OneLogin, PingFederate, FusionAuth or any SAML compliant Identity Provider. Joomla SAML Single-Sign-On addresses the challenge of maintaining the credentials for each application separately, streamlining the process of signing on without the need to re-enter the password.
  • Using Discourse
    • Editing Titles in Discourse - support - Discourse Meta
      • What’s the deal with people being able to edit titles? Can only Regulars do this? I ask because on another forum, we are having some issues with people editing topic titles.
      • Because you’re a longtime, trusted, “Regular”, you get elevated privileges, so you can fix bad titles.
Published in Web Apps and Software
Read more...
Wednesday, 01 May 2024 14:38

Forward Proxy, Reverse Proxy, TrueNAS, Virtualmin and Discourse

Suggested alternative Titles - might replace - current one

  • Run a TrueNAS Docker container through Virtualmin by using it as a Reverse Proxy
  • Run Discourse on the internet from a TrueNAS Docker container, through a Virtualmin Reverse Proxy
  • Run Discourse on the internet through a Virtualmin Reverse Proxy from a TrueNAS Docker container
  • Run Discourse on the internet through a Virtualmin Reverse Proxy using a TrueNAS Docker container
  • Run Discourse through a Virtualmin Reverse Proxy using a TrueNAS Docker container
  • Run Discourse through a Virtualmin Reverse Proxy using a TrueNAS Docker container as the source

 

Terms block

  • WebSocket
  • WebSocket Proxy
  • :80 --> ws://
  • :443 --> wss://
  • Reverse Proxy
  • Forward Proxy / Gateway Proxy
  • Unix Socket
  • ProxyPass
  • ProxyPassReverse

 

Setup instructions

  • Setup truenas
  • setup Discourse Docker
  • setup virtualmin
  • setup virtual server
  • setup proxying
  • bridge wordpress/Joomla with discourse using a SSO (single Sign On)

 

 

This article will explore Forward Proxies, Reverse Proxies with a practical example to set up Virtualmin to access a remote Discourse server. We will also use phpmyAdmin to show case a locally hosted App. I will be using Apache but nginx can act as a reverse proxy.

Technologies used in this tutorial

  • Ubuntu
  • Webmin/Virtualmin
  • Apache
  • ProxyPass / ProxyPassReverse
  • Discourse
  • Docker
  • TrueNAS

Proxying allows you to map a URL path in a virtual server's website to one or more destination URLs. It can be used to make other services available via a URL path on your website. For example, you might map the path /nodejs to the URL http://localhost:3000, an instance serving the application running on your Virtualmin system.

A proxy maps some URL on a virtual server to another webserver. This means that requests for any page under that URL path will be forwarded to the other site, which could be a separate machine or another webserver process on the same system.

  • General
    • A reverse proxy is a server that sits between a client and a web server, acting as an intermediary for requests. It retrieves resources on behalf of the client from the server, and then returns the response to the client.
    • Proxy Paths, 2 Ways
      • sockets = good when all sites are on the same site, a lot mor esecure and quicker (unix sockets?)
      • external ?? = when not on  same server, a little less secure and probably slower.
    • phpMyadmin App (local example)
      • install phpmyadmin on its own domain sub domain on your primary domain.
      • use proxypaths to redirect a clients domain here. also you can use unix sockets.
      • use the server templates to install this.
      • add this section when i have done proxypaths. I might be able to get www.example.com/phpmyadmin path (not whole domain) to work
    • Installation Discussions
      • Installing Discourse Forum on Virtualmin - #4 by Joe - Virtualmin - Virtualmin Community
        • I’ve discussed this in the past, as well. But, the most important thing is: 
          ProxyPass / unix:/var/discourse/shared/socket-only/nginx.http.sock|http://localhost/
ProxyPassReverse  / unix:/var/discourse/shared/socket-only/nginx.http.sock|http://localhost/
        • Note the socket can be wherever you put the Discourse directory, including in a domain owner home directory. This is just the path you’d use if installing Discourse exactly according to the docs, and then changed Docker to listen on a socket rather than directly on an IP and port 80/443 (which can’t happen on your Virtualmin system because Apache or nginx is already there).
        • And, as always, if you want to use Let’s Encrypt in Virtualmin or via certbot using web validation, you’d also need to exclude the .well-known path, and let it be served from the file system. Again, this is always true, nothing special about Discourse or Virtualmin being involved.
      • I want to install using a reverse proxy for Apache - installation - Discourse Meta
  • Different Types of Proxies
    • ? Forward Proxy = forwards requests to the new domain/target. The user is semi aware of the redirection. (i.e. the browser is aware, this fact is hidden from the user).
    • ? Reverse Proxy - web server requests the information for you and then supplies it to the user. the user never sees the secodn server.
    • Forward proxy vs. reverse proxy: What's the difference? | TheServerSide
      • Great article
      • Despite similarities, the use case for forward proxy vs. reverse proxy are vastly different. Here are key differences between a forward and reverse proxy.
      • A forward proxy accepts connections from computers on a private network and forwards those requests to the public internet. It is the single point of exit for subnet users who want to access resources outside of their private network.
      • As the name implies, a reverse proxy is the opposite of a forward proxy. The reverse proxy acts as a single point of entry for external systems to access resources on a private subnet.
    • Forward Proxy vs. Reverse Proxy: The Difference Explained - In this post we dissect the differences between proxy & reverse proxy and explain how admins can use a reverse proxy for easy access management control.
    • webserver - What's the difference between a proxy server and a reverse proxy server? - Stack Overflow
    • Proxy vs Reverse Proxy - What's the Difference? (Pros and Cons)
      • Proxy vs Reverse Proxy – What’s the Difference? (Pros and Cons).
      • A proxy server is one of the tools that help protect a network and the assets that are connected to it. They make sure users, data packets and computers are safe and can communicate effectively.
      • We will have a look at Proxy vs Reverse Proxy, make a comparison to distinguish which one should be used, their use cases and pros and cons for each proxy.
    • Proxy vs Reverse Proxy: Understand the Difference - Netnut - Proxy vs Reverse Proxy: An in-depth comparison detailing their unique roles, benefits, and usage in network security and performance enhancement.
    • Reverse Proxy Guide - Apache HTTP Server Version 2.4
      • In addition to being a "basic" web server, and providing static and dynamic content to end-users, Apache httpd (as well as most other web servers) can also act as a reverse proxy server, also-known-as a "gateway" server.
      • In such scenarios, httpd itself does not generate or host the data, but rather the content is obtained by one or several backend servers, which normally have no direct connection to the external network. As httpd receives a request from a client, the request itself is proxied to one of these backend servers, which then handles the request, generates the content and then sends this content back to httpd, which then generates the actual HTTP response back to the client.
      • There are numerous reasons for such an implementation, but generally the typical rationales are due to security, high-availability, load-balancing and centralized authentication/authorization. It is critical in these implementations that the layout, design and architecture of the backend infrastructure (those servers which actually handle the requests) are insulated and protected from the outside; as far as the client is concerned, the reverse proxy server is the sole source of all content.
  • Virtualmin (General)
    • Proxy Paths - Is this Pro - Virtualmin - Virtualmin Community
      • It is not a bug. It moved into GPL with the most recent version.
      • I wanted it to be more clear that you can use app servers and Docker/Podman in Virtualmin, and easier to explain how to do that. Since the way you use app servers or Docker/Podman is to proxy to it. So, there it is. Nobody can imagine it’s complicated anymore (but, they will anyway, as a lot of folks don’t know that the way you use Docker in any hosting deployment is to proxy to it).
      • Proxy Paths is what’s new in GPL. You could always do proxying in GPL, it just wasn’t obvious how for users unfamiliar with proxying in Apache. You could, at any point in the past, setup a Docker container, or whatever app server, and setup proxy rules to make the app available on a path, but we’ve had so many people yell at us about not being able to run whatever random crap they want to run, that I wanted it to be as simple as we could make it, so people would stop saying, “You can’t use Docker with Virtualmin”. (They won’t stop saying it, but at least I tried.)
      • You don’t need to delve into any of the other proxy-related stuff (unless you want to), you only need to setup Proxy Paths for your apps. None of those other forms are relevant to this new feature, and it shouldn’t require too much in the way of documentation to use it, though I think the naming of the options is weird…it should also have popup help, I’m not sure why I missed that it didn’t.
      • Anyway, this is just about making a local app server or docker container visible to the world through a proxied path, in a very simple form. The Pro feature was always about making it simple, not about a thing that was impossible in GPL. I’ll try to make sure the next release has help files, though, so we’ve got some popup help for these options.
      • Q: should there not be a one for http and one for https?
      • A: No. http is dead. Don’t treat them differently, or you’ll break your website for some browsers. The only thing you should ever do with http is redirect it to https (and some browsers are going to do that automatically anyway).
    • Basic Reverse Proxy? Virtualmin - Virtualmin Community
      • Ok, so there are actually two different proxy sections under server configuration, ‘Edit Proxy Website’ and ‘Proxy Paths’.
      • ‘Proxy Paths’ seems to invoke mod_proxy_balancer, which is actually something I would like to do in the future, but still can’t get it to do SSL.
      • However ‘Edit Proxy Website’ is very straightforward and is working as one would expect so that’s good enough for now.
    • Squid Proxy Server | Webmin
      • This article explains what an HTTP or FTP proxy server is, and then explains how Webmin can be used to configure the popular Squid proxy server.
      • An HTTP proxy server is basically a program that accepts requests from clients for URLs, fetches them on behalf of the client, and returns the results to the client. Proxies are used on networks where clients do not have direct access to the Internet but still need to be able to view web pages. A proxy is also used for caching commonly requested pages so that if more than one client wants to view the same page it only has to be downloaded once.
  • Proxy setting locations in Virtualmin
    • Virtualmin --> Web Configuration --> Edit Proxy Website
      • This form can be used to enable and configure the URL to which all requests to this virtual server's website are forwarded.
      • This will proxy the whole website/domain.
    • Virtualmin --> Web configuration --> Configure Website |Configure SSL Website --> Proxying
      • This will allow you to configure Proxying of the website with more control than the above option.
    • Virtualmin --> Web Configuration --> Proxy Paths
      • Proxying allows you to map a URL path in a virtual server's website to one or more destination URLs. This is typically used to make Ruby on Rails applications served by multiple Mongrel instances available under a URL path in the server's website. It can also be used to make other webservers such as Tomcat available via a URL path on your website.
      • For example, you might map the path /radiant to the URLs http://localhost:3001 and http://localhost:3002, which are the ports for two Mongrel instances serving the same application running on your Virtualmin system.
      • This is now the preferred method to setup proxying.
      • This has been moved from the Pro version.
      • You can proxy the whole website or just individual directories
  • Virtualmin errors people had
  • Virtualmin Reverse Proxy Tutorials
  • General Reverse Proxy Tutorials
  • Discourse General
    • WP Discourse – WordPress plugin | WordPress.org - This plugin allows you to use DiscThe WebSocket API (WebSockets) - Web APIs | MDN - The WebSocket API is an advanced technology that makes it possible to open a two-way interactive communication session between the user's browser and a server. With this API, you can send messages to a server and receive event-driven responses without having to poll the server for a reply.ourse as a community engine for your WordPress website.
    • What forum software used by Virtualmin? - #2 by Joe - Site Feedback - Virtualmin Community
      • It’s self-hosted Discourse 4. I’ve talked about it a bunch, including a couple/few News posts.
      • It has only minor customizations, mostly to styles. We use a couple of plugins, Solved for marking questions solved, and Akismet (with a paid account) for spam filtering.
      • As the admin/maintainer of it, I love it, and I think most users are pretty happy with it. It integrates relatively well with WordPress using an off-the-shelf plugin, Ilia added a bit of code so we can see people’s license information on their posts (since only Pro customers can request private support via PM, we like to be able to refer GPL users to the public forums without having to check their account in WordPress).
      • Akismet is only average at catching spam, but the basic Discourse auto-moderation tools work really well. The built-in stuff catches far more spam than Akismet does, actually, but combined they add up to a very effective system.
      • It’s also reasonably fast, though could always be faster.
      • Compared to the old Drupal forums we cycled through (and the forums before it on Joomla and OpenACS), Discourse is a dream. I have very few complaints. The way it deals with Tags is incompatible with my brain, and last I checked it still had a bug that meant I can’t let others create Tags. But, that’s so minor, and none of the previous forums even had tags (and I had to do a lot more customization and bugfixes to the features we were using). Discourse is so remarkably better than any forum I’ve ever used (which is a lot of them; five or six for Virtualmin alone) it feels petty to complain about anything.
  • Discourse Chat Overview
    • Discourse Chat is now available. - Chat is here! Enable conversations in your community to flow between faster-paced chat and slower-paced discussions.
    • An overview of features in Discourse Chat - users - Discourse Meta - You may have noticed Chat recently being added to your community, or here on Meta. In this topic we’ll take a look at the various features included in Chat, and how you can make the most of them!
    • Discourse Chat Integration Plugin - plugin - Discourse Meta
      • As this is an official plugin maintained by the Discourse team, support, bug, ux, and feature requests can be made in the respective categories here on Meta, and tagged with the appropriate plugin tag.
      • discourse-chat-integration allows sending notifications about new Discourse posts to ‘group chats’ on a number of instant messaging platforms.
      • Notifications can be triggered by new topics, new replies, messages to a group, or mentions of a group.
    • Official Discourse Chat Plugin - plugin - Discourse Meta
      • This plugin is bundled with Discourse core.
      • There is no need to install this plugin separately.
      • As mentioned in our initial announcement and most recent update, this plugin adds chat functionality to your Discourse so it can natively support both, long-form and short-form communication needs of your online community.
  • Discourse Install Tutorial
    • How Do I Install Discourse? | Discourse Docs - GitHub
      • The only officially supported installs of Discourse are Docker based.
      • You must have SSH access to a 64-bit Linux server with Docker support.
      • We regret that we cannot support any other methods of installation including cpanel, plesk, webmin, etc.
  • Discourse proxying setup errors
  • Docker vs Podman
    • Virtualmin app server plugin · Issue #626 · virtualmin/virtualmin-gpl · GitHub
      • Yeah, podman is generally considered recommended across the Linux distro landscape, because Docker hasn't been super friendly to Open Source. And, technically, podman is superior on several fronts (no daemon BS, not tied to any commercial cloud services, uses a bunch of more modern Linux security features). But, and this is a big but, nobody knows the name "podman", and everybody wants "Docker support". So, I guess we need to detect if the user has the docker tools installed (from third party sources, probably) and allow use of those instead. Because podman doesn't use a daemon, it isn't entirely compatible with Docker on the same system; so, even though they have almost identical command line options and use the same Dockerfile syntax, if you list containers with podman you won't see the ones Docker is managing and vice versa, AFAIK, and you can't stop/start/pull/push/etc. across the two. I prefer podman, and I hope most users who don't know the difference will just accept it.
  • WebSockets

 

 

 

Published in Web Server
Read more...
Wednesday, 14 February 2024 11:42

What is CGI, FastCGI and PHP-FPM

I have struggled for years to understand  what the point of the cgi-bin folder was for which then lead me to research into the whole area and what was the point of this technology.

What is CGI?

  • CGI stands for 'Common Gateway Interface'
  • CGI is a standard for external gateway programs to interface with information servers such as HTTP servers.
    • Or in other words, programs in other languages can be used and utilised via HTTP requests.
    • A way to run a server side script (PHP, Perl, Python,...) when a HTTP request comes.
  • This allows Binaries and scripts of other programming languages to be run such as Python and C programs.
  • Currently maintained by the NCSA (National Center of Supercomputing Applications).
  • CGI has been around long before PHP.

What is the purpose of CGI?

CGI allows the use of scripts and binaries from many other languages and the benefit of this are:

  • You can utilise these languages and their inherent feature sets directly from a website.
  • Different languages have different abilities
  • Programming is not limited to the web-based languages.

`cgi-bin` folder, why do I have this on my web server?

This is the only folder allowed on your server where you can runs CGI applications and is usually created when you install the package `php-cgi`.

How Does it work?

  • Place scripts or binaries in the folder /cgi-bin/ with the file extension .cgi, other might work such as .py but I have not tried this.
  • Run the script accessing the relevant URL such as https://example.com/cgi-bin/mytest.cgi
  • The Script:
    • is started
    • reads the Stdin (Standard in) and Environmental variables as required
    • processes the data
    • outputs the response to the Stdout (Standard Out)
    • is terminated
  • The webserver returns the Stdout as the http response
  • The file extension .cgi can be changed if required when setting up the server.
  • The .cgi files can be either scripts or binaries.
  • A CGI can only external interact externally as follows (but this does not include what its internal routines do):
    • Read the Stdin
    • Read the Environmental Variables
    • Output the Stdout
  • When a CGI App is started a single process for it is created and when it has finished executing the process is terminated removing it from RAM etc.
  • You can easily upgrade a CGI script by just changing the CGI file.

Running PHP via CGI, FastCGI, PHP-FPM

PHP can be run via any of the following wrappers but this is not the same for CGI Apps

  • mod_php
    • Old, insecure and slow.
    • Never use this.
    • This is the original way to run PHP and is here for reference only.
  • CGI
    • PHP runs as a CGI App
    • Opens a single process for each request, runs it and then when finished, is immediately closed.
  • FastCGI
    • PHP runs as a FastCGI App
    • Maintains a pool of workers for running scripts
    • FastCGI can use a single persistent process which handles many requests over its lifetime.
  • PHP-FPM
    • FastCGI Process Manager (FPM)
    • FPM (FastCGI Process Manager) is a primary PHP FastCGI implementation containing some features (mostly) useful for heavy-loaded sites.
    • Only runs PHP, does not allow for traditional CGI or FastCGI apps.
    • This now the recommend method for PHP and is by far the fastest.
    • This is know as PHP-FPM rather than just FPM because it only runs PHP.
    • The Linux package name is php-fpm

Notes

  • CGI is one process for each request and has large overheads.
  • FastCGI
    • uses a persistent process and requires more settings for this.
    • is a bit more complicated because the processes start up at the very beginning of the server.
  • FastCGI keeps a pool of scripts running whereas CGI opens a single process, runs it and then when finished is immediately closed. Because the processes are always open there is no overhead of starting up the process any more.
  • CGI and FastCGI Apps are not the same, but are similar. FastCGI Apps need extra code to handle persistent states.
  • You can run a CGI App under FastCGI but you need a wrapper app.
  • CGI and FastCGI are almost always run from the cgi-bin folder
  • The file extensions of .cgi and .fcgi are use as appropriate

CGI and FastCGI are getting replaced

These technologies are getting or already have been replaced with proxying and dedicated server modules such as mod_perl and mod_python.

Most programming languages offer their own dedicated servers so it makes sense to have those do the relevant work and then return the results to your web server. Proxying allows a web server to do this invisibly just as CGI and FastCGI have done in the past and allows the resource hit to be offloaded.

Proxying is configured in your Apache Config files.

What about PHP-FPM?

This technology is dedicated to running PHP, and in a sense is probably now a dedicated engine for PHP only.

I cant see this going away anytime soon but it will only ever run PHP.

Virtualmin

I have put my Virtualmin notes here as they are more relevant to this niche feature.

  • Virtualmin only supports CGI Apps out of the box. You can manually add FastCGI support.
    • Virtualmin installs mod_fcgi, because it is a dependency of the php extension php-fpm which is installed by default.
    • mod_fcgi requires you to set the correct handlers in the Apache conf and these are not set by Virtualmin.
    • Virtualmin supports CGI Apps out of the box as and has GUI pages for this.
    • Your PHP mode does not affect the running of CGI Apps, they are separate.
  • Does the php-cgi module include cgi and fastcgi
    • no, php-fpm includes FastCGI and PHP-FPM
    • php-cgi includes only CGI
  • GUI Locations
    • Webmin --> Servers --> Apache Webserver --> Global Configuration --> CGI Programs
    • Webmin --> Servers --> Apache Webserver --> Existing virtual hosts --> select virtual server --> CGI Programs
    • Virtualmin --> Web configuration --> (Configure SSL Website | Configure Website) --> CGI Programs
  • Changing the PHP mode does not change whether the server is going to run CGI or FastCGI Apps, it only alters how PHP s run.
  • Should ExecCGI be removed from the public_html folder apache directive? - Virtualmin - Virtualmin Community
    • Q:
      • My understanding is that ExecCGI is the command that can enable or disable the running of a CGI.
      • In the code below it looks like the apache directive is allowng the enabling of CGI Apps in the root of the public_html folder.
      • So my question is, should ExecCGI be removed from the public_html folder or have I missed something, perhaps this is required for PHP to be run? 
        <Directory /home/example/public_html>
    Options -Indexes +IncludesNOEXEC +SymLinksIfOwnerMatch +ExecCGI
    Require all granted
    AllowOverride All Options=ExecCGI,Includes,IncludesNOEXEC,Indexes,MultiViews,SymLinksIfOwnerMatch
</Directory>
<Directory /home/example/cgi-bin>
    Require all granted
    AllowOverride All Options=ExecCGI,Includes,IncludesNOEXEC,Indexes,MultiViews,SymLinksIfOwnerMatch
</Directory>
    • A:
      • AllowOverride means .htaccess can override the default configuration. The default configuration does not have ExecCGI.
      • You can, of course remove it, if you don’t want to allow that to be overriden by .htaccess. Historically it was pretty common to run CGI scripts in public_html, too, but much less so these days.
      • Presumably you allow execution of PHP in public_html, if you’ve got any web apps installed. So, if a remote attacker gains ability to write to public_html, it’s already over. There would be no need to modify .htaccess to run scripts, they’d just drop a PHP shell in public_html and be done.
  • Does Virtualmin support FastCGI Apps - Virtualmin - Virtualmin Community
    • Q: I know Virtualmin supports standard CGI Apps, but does it support FastCGI Apps?
    • A:
      • Apache (and nginx) support FastCGI. Virtualmin isn’t your web server.
      • It generally needs some app-specific configuration, which kinda depends on what language(s) and app server(s) you’re using and how you’re using them. The things Virtualmin knows how to configure with FastCGI it does so (Install Scripts might use FastCGI, for example, and that will get configured). There is no fcgi-bin directory, though, by default, so there’s nowhere you can just drop a FastCGI application and expect it to magically run.
      • The necessary Apache module (mod_fcgid) is installed by default, though. mod_fcgid - FastCGI interface module for Apache 2 - The Apache HTTP Server Project
  • Best selection of PHP extensions - #13 by Jamie - Virtualmin - Virtualmin Community
    • php-fpm is independent of running PHP apps via fcgid+suexec (which uses php-cgi and predates the existence of php-fpm). FPM is an application server for PHP apps that speaks the FastCGI protocol. FastCGI is a protocol, not any specific implementation, and there are several ways to run Apache+PHP apps using the FastCGI protocol.
    • You should use FPM, if you can. The implementation is much simpler (from the user perspective), it is much more actively maintained, it is more efficient, etc. And, it provides the same security benefits of suexec.
  • Can I remove the system default CGI PHP
    • php-cgi has nothing to do with the virtualmin miniserv which runs on Perl.
    • The CLI version of PHP has nothing to do with CGI.
  • Can I safely remove CGI on my alternative php versions?
    • yes

Links

  • CGI
  • FastCGI
  • CGI vs FastCGI vs PHP-FPM
    • How does Fast CGI work? - YouTube - A simple video but explains the difference well
    • Fast CGI - YouTube | Philip Bohun
      • A quick video describing fast CGI, how it differs from CGI, as well as its pros and cons.
    • PHP: CGI and command line setups - Manual | php.net
      • By default, PHP is built as both a CLI and CGI program, which can be used for CGI processing.
    • PHP CGI and FPM, what are they? - Stack Overflow
    • PHP: FastCGI Process Manager (FPM) - Manual | php.net
    • php fpm - Differences and dis/advanages between: Fast-CGI, CGI, Mod-PHP, SuPHP, PHP-FPM - Server Fault
    • CGI vs FastCGI vs PHP-FPM. CGI: CGI stands for Common Gateway | by Miladev95 | Medium - CGI:CGI stands for Common Gateway Interface.- It is a standard protocol that defines how web servers can interact with external
      • CGI
        • While CGI was revolutionary in its time and enabled dynamic web content, it has certain limitations, including performance overhead and scalability issues. As a result, modern web development has largely moved away from CGI in favour of more efficient and scalable server-side scripting technologies like PHP, Node.js, and various application frameworks.
        • Today, CGI is mostly used in specific cases where compatibility with legacy systems or specialized environments is required. For general-purpose web development, more efficient alternatives are preferred.
        • In summary, CGI is the simplest and oldest method for executing server-side scripts but suffers from performance overhead. FastCGI improves upon CGI by introducing a persistent process pool, resulting in better performance. PHP-FPM, on the other hand, is a specific FastCGI implementation tailored for PHP execution, providing superior performance, scalability, and security for serving PHP applications. For modern PHP applications, PHP-FPM is the recommended method for handling dynamic content.
      • FastCGI
        • FastCGI is widely used in web server configurations to handle dynamic content generation efficiently. Popular web servers like Nginx and Apache support FastCGI, making it a standard choice for hosting dynamic web applications written in languages like PHP, Python, Ruby, and more.
        • In summary, FastCGI is an extension of CGI that significantly improves the performance and resource utilization of web applications by employing a persistent application process pool to handle incoming requests.
      • PHP-FPM
        • Overall, PHP-FPM is a valuable tool for PHP application deployment, providing improved performance, security, and scalability for serving PHP-driven web applications.
      • In summary, CGI is the simplest and oldest method for executing server-side scripts but suffers from performance overhead. FastCGI improves upon CGI by introducing a persistent process pool, resulting in better performance. PHP-FPM, on the other hand, is a specific FastCGI implementation tailored for PHP execution, providing superior performance, scalability, and security for serving PHP applications. For modern PHP applications, PHP-FPM is the recommended method for handling dynamic content.
    • What are CGI and fast CGI - This is part of the Semicolon&Sons Code Diary - consisting of lessons learned on the job. You're in the web-development category.
    • Which PHP mode? Apache vs CGI vs FastCGI - Layershift Blog - There are multiple ways to execute PHP scripts on a web server. We’re often asked about the difference between these modes, so here it is!
    • Difference between PHP-CGI and PHP-FPM | BaseZap - When running PHP through the web server, there are two distinct options: running it using PHP's CGI, or running it as a PHP-FPM, for the web server.
    • What is difference between PHP cli and PHP cgi? - Stack Overflow
      • Basically it's a way to run a server side script (PHP, Perl, Python,...) when a HTTP request comes.
      • Excellent descriptions.
    • CGI, FastCGI and PHP-FPM - Some basic questions - Help! (Home for newbies) - Virtualmin Community
      • Q: Is running CGI Apps now considered an old way of doing things?
      • A: Generally, yes. For large applications, having them load all their dependencies on every run is wildly inefficient. For tiny little apps that don’t load a bunch of extra junk, it may be fine. e.g. if you just need to serve out one tiny API endpoint and you can write a Perl/Python/Ruby/PHP script that runs in a few milliseconds, then it may make sense to use CGI (but an app server probably isn’t notably worse than CGI in those cases, either…it’d be about balancing CPU vs. memory usage).
      • Q: FastCGI - allows CGI Apps to be run in cgi-bin
      • A: No. FastCGI can not run CGI scripts. It requires applications built to communicate via the FastCGI protocol, which is not the CGI protocol. A CGI script is not a FastCGI script without a wrapper. We provide a wrapper (fcgiwrap), so you can run CGI applications on systems that don’t have suexec available, but it does not mean FastCGI is running CGI scripts. fcgiwrap runs CGI scripts and communicates with the web server via the FastCGI protocol. It is the application server in this context.
      • Q: But, this is a bit misleading: “Virtualmin only supports CGI Apps out of the box. You can manually add FastcGI support.”
      • A:
        • Virtualmin supports, with full UI support, PHP apps via many execution modes, including FastCGI, and multiple versions of PHP (when installed). Virtualmin also supports setting up proxy rules to any other kind of application server (many app servers do not use FastCGI, they just use HTTP, and that is the most common way to run most apps other than PHP at this point in history). FastCGI is on the way out, too.
        • We brought proxy paths support from Pro into GPL just recently to ease that use case. And Virtualmin Pro is about to get app server and container-based app support (where it has some awareness of the app server or docker/podman and configures the proxy rules automatically).
        • So, nearly any way you want to run apps in Virtualmin is very easy to do.
  • Old Technology
    • How to run PERL scripts - #7 by Joe - Help! (Home for newbies) - Virtualmin Community
      • You should be moving your apps to a modern execution mode. Plack is probably the easiest way to migrate CGI Perl scripts to something modern. https://plackperl.org/
      • Q: Do I need Plack, if I use Apache?
      • A:
        • CGI is the slowest way to run a web app, so the web server is completely irrelevant in your deployment, so I recommend you use the one that is most commonly used for CGI scripts, which is Apache.
        • You don’t need Plack. I was offering you a solution for improving the way you’re executing your apps. Plack is a low-friction path from CGI to running apps under a long-running Perl app server (this saves significant startup time for each request). If you use Plack, you no longer need CGI script support in your web server or a wrapper (because it is the wrapper and it provides a fastcgi interface you can proxy to). If you were to use Plack, you would be able to use nginx. Performance still wouldn’t matter, but at that point I would recommend whichever web server you’re most comfortable with rather than having a relatively strong opinion that Apache is the better choice.
        • I don’t know anything about the app you’re running; there’s nothing I can tell you about it. But, I know how CGI works with Apache and how it doesn’t work with nginx (unless you wrap it). So, if you’re running CGI scripts, you should go the path of least resistance, unless you have a good reason to do otherwise (performance is not a good reason, because the web server cannot save you from CGIs relative poor performance).
  • Programming
  • Useful Examples
  • Run CGI Apps on FastCGI
    • fcgiwrap
      • simple server to run CGI applications over FastCGI
      • fcgiwrap is a simple server for running CGI applications over FastCGI.
      • Its goal is to provide clean CGI support to the nginx webserver, although can be used with others.
      • fcgiwrap is lightweight and has no configuration, making it possible to use the same pool to run different sites.
    • How to execute CGI scripts using fcgiwrap | sleeplessbeastie's notes - I am an enthusiast of the uWSGI project. You are still not limited to it as you can use fcgiwrap, a very lightweight and straightforward FastCGI wrapper for CGI scripts that do not require additional configuration.
    • GitHub - leejo/cgi-fast
      • The new home for CGI::Fast, removing it from the original CGI.pm distribution - leejo/cgi-fast
      • CGI::Fast is a subclass of the CGI object created by CGI.pm. It is specialized to work with the FCGI module, which greatly speeds up CGI scripts by turning them into persistently running server processes. Scripts that perform time-consuming initialization processes, such as loading large modules or opening persistent database connections, will see large performance improvements.
    • FcgiWrap - Community Help Wiki
      • fcgiwrap is a simple server for running CGI applications over FastCGI. It hopes to provide clean CGI support to Nginx (and other web servers that may need it).
      • fcgiwrap can be used together with Nginx to serve CGI or Perl scripts (.cgi).
    • How to execute CGI scripts using fcgiwrap | sleeplessbeastie's notes - I am an enthusiast of the uWSGI project. You are still not limited to it as you can use fcgiwrap, a very lightweight and straightforward FastCGI wrapper for CGI scripts that do not require additional configuration.
  • Alternatives
    • ipc - Are there alternatives to CGI (and do I really need one)? - Stack Overflow
      • Even under moderate loads, CGI is a pretty unscalable beast. FastCGI is an option, but you'll probably also find a mod_XXXX package where XXXX is the name of your language. There's a mod for ruby, perl, and python for instance and probably a fair few others.
    • Is CGI still a thing? Which alternatives are there? | Reddit
      • I stumbled upon CGI and tried to set it up on a server – provided by uberspace.de – but failed. Their support told me that CGI isn’t too relevant for them anymore, as basically any language has its own webserver nowadays and that I should just run the one I want as a service listening to a specified port
      • CGI was a way of running a specific script through an http request.

 

 

 

 

 

Published in Web Server
Read more...
Thursday, 08 February 2024 18:10

PHP MySQL Database Drivers, Extensions and APIs

Deciding on how to connect your software to the relevant database requires you to understand the relevant technologies. I struggled with this myself so I looked it up and made some notes.

TL;DR

  • Modern version of PHP are compiled by default with php-mysql which has in it:
    • mysqlnd
    • mysqli
    • pdo_mysql
  • Use PDO for your new PHP projects.

What I found

After going thourh my research I put together this list of answers and points.

  • php-mysql
    • This is a PHP extension and ia a binary file.
    • This provides the following API Extensions for PHP:
      • MySQLi
      • MYSQL PDO
      • MySQL - This is no longer provided
    • This provides the following extensions (in a normal build):
      • mysqlnd
      • mysqli
      • pdo_mysql
    • Compile notes
      • This can be compiled to use the 'MySQL Native Driver' (mysqlnd) or the MySQL Client Library (libmysqlclient - the external shared driver from Oracle).
      • Since PHP 5.6 MySQL Native Driver is the default at compile time.
      • If compiled with mysqlnd you will see this extension in the list of 'Compiled in modules'.
      • From PHP 8.3 onwards, the ability to compile with libmysqlclient has been removed.
    • This does not provide the MySQL server.
    • This extension works for MySQL and MariaDB.
    • I think that using mysqlnd and MySQL PDO will allow you to to talk to other SQL servers.
  • Legacy PHP Extensions
    • The following PHP extensions no longer exist as separate binaries:
    • I am not sure in which version of PHP they were removed, but they are now aliases (Virtual Packages) of php-mysql and they are now included directly in this binary.
  • libmysqlclient
    • A shared driver from Oracle.
    • MySQL database client library.
    • Ubuntu – Details of package libmysqlclient21 in jammy
    • This was an external library that the PHP extensions php-mysql and php-mysqli used to talk to SQL servers to allow their APIs they presented to PHP to work.
    • This is not a PHP extension but an external library
  • MySQL Native Driver (mysqlnd)
    • This is the PHP team's implementation of the MySQL API layer and requires no external libraries.
    • The MySQL native driver for PHP (mysqlnd) is a drop-in replacement for the MySQL Client Library (libmysql) for the PHP script language.
    • This has a lot of improvements over using the libmysqlclient library.
    • This no longer exists as a separate PHP extension.
    • This is now the default driver in the php-mysql extension.
  • Ubuntu
    • When the PHP extensions are installed they are automatically enabled.
    • You can enable, disable and view the status of PHP extensions with the following commands:
      • phpenmod – Used to enable modules in PHP
      • phpdismod – Used to disable modules in PHP
      • phpquery – Used to view status of modules of PHP
  • Windows (Xampp)
    • To enable or disable extensions you need to edit the php.ini and comment/uncomment the relevant line in the Dynamic Extensions section.
    • You might see the legacy extension here and that is fine as these extensions are compiled for Windows which are not always the latest (or there might be other reasons).
  • Mysql vs MySQLi vs PDO
    These are called APIs Extensions and are not separate binaries anymore.
    • PDO
      • Can work with many different databases
      • OOP only
      • Supports prepared statements
      • Current and secure
      • Use this for all your new Projects
      • This API also adds its own abstraction layer to allow communication to the different types of databases.
      • You can wrote code once that will work on different databases with little or no alterations.
    • MySQLi
      • Only works with MySQL and MariaDB
      • Procedural and OOP
      • Supports prepared statements
      • Current and secure
    • MySQL
      • Only works with MySQL and MariaDB
      • Procedural only
      • Doesn't support prepared statements
      • Legacy and insecure
      • Removed in PHP7
      • Don't use
  • CMS
    • WordPress
      • Only uses MySQLi
    • Joomla
      • Has a database abstraction layer so it can use both MySQLi and PDO

Research

What are they and what do they do? Which is best for performance

  • php-mysqlnd
    • The following command will install php-mysql because php-mysqlnd is an alias.
      sudo apt install php-mysqlnd
    • MySQL native driver for PHP
    • There is no extension file for this in the Ubuntu repo.
    • MySQL :: MySQL native driver for PHP - mysqlnd
      • The MySQL native driver for PHP (mysqlnd) is a drop-in replacement for the MySQL Client Library (libmysql) for the PHP script language.
      • The PHP MySQL extensions are lightweight wrappers on top of a C client library. There are 3 PHP MySQL extensions:
        1. ext/mysql (not recommended)
        2. ext/mysqli
        3. PDO_MySQL
      • The extensions can either use the mysqlnd or libmysql library to connect from PHP to MySQL. Choosing one or the other library is a compile time decision. Both libraries are supported and constantly being improved.
      • MySQL recommends using the MySQL native driver for PHP (mysqlnd) together with ext/mysqli or PDO_MySQL.
      • As of PHP 5.4, the mysqlnd library is a php.net compile time default to all PHP MySQL extensions. Also, the php.net Windows team is using mysqlnd for the official PHP Windows distribution since mysqlnd became available in PHP 5.3.
    • Mysqlnd - MySQL Native Driver - Manual | PHP Manual (php.net)
      • Introduction
        • MySQL Native Driver is a replacement for the MySQL Client Library (libmysqlclient). MySQL Native Driver is part of the official PHP sources as of PHP 5.3.0.
        • The MySQL database extensions MySQL extension, mysqli and PDO MYSQL all communicate with the MySQL server. In the past, this was done by the extension using the services provided by the MySQL Client Library. The extensions were compiled against the MySQL Client Library in order to use its client-server protocol.
        • With MySQL Native Driver there is now an alternative, as the MySQL database extensions can be compiled to use MySQL Native Driver instead of the MySQL Client Library.
        • MySQL Native Driver is written in C as a PHP extension.
      • Overview
        • Although MySQL Native Driver is written as a PHP extension, it is important to note that it does not provide a new API to the PHP programmer. The programmer APIs for MySQL database connectivity are provided by the MySQL extension, mysqli and PDO MYSQL. These extensions can now use the services of MySQL Native Driver to communicate with the MySQL Server. Therefore, you should not think of MySQL Native Driver as an API.
        • Using the MySQL Native Driver offers a number of advantages over using the MySQL Client Library.
        • Because MySQL Native Driver is written as a PHP extension, it is tightly coupled to the workings of PHP. This leads to gains in efficiency, especially when it comes to memory usage, as the driver uses the PHP memory management system. It also supports the PHP memory limit. Using MySQL Native Driver leads to comparable or better performance than using MySQL Client Library, it always ensures the most efficient use of memory. One example of the memory efficiency is the fact that when using the MySQL Client Library, each row is stored in memory twice, whereas with the MySQL Native Driver each row is only stored once in memory.
        • Reporting memory usage: Because MySQL Native Driver uses the PHP memory management system, its memory usage can be tracked with memory_get_usage(). This is not possible with libmysqlclient because it uses the C function malloc() instead.
        • MySQL Native Driver also provides some special features not available when the MySQL database extensions use MySQL Client Library.
    • Easy Read/Write Splitting with PHP’s MySQLnd
      • This gives a bit of the history about this extension
      • MySQL Native Drive (mysqlnd) was added in PHP 5.3 and has been the default since PHP 5.4 (though you can still compile against libmysqlclient). It brings extra features, better performance, and better memory usage than libmysqlclient.
    • How to Install MYSQLND: mysqli Extension for php in Ubuntu? [SOLVED] - TheQuickBlog
      • The mysqli extension is missing. Please check your PHP configuration. See our documentation for more information.
      • this shows installation command using 
        sudo apt install php7-mysqlnd
    • PHP: What is mysqlnd, do I need it? | 
      • Mysqlnd is neither a new PHP extension nor a new API! mysqlnd is new C-level library code. The mysqlnd library provides almost the same functionality as libmysql does. Both C-libraries implement the MySQL communication protocol and can be used to connect to the MySQL Server.
      • mysqlnd has been designed as a drop-in replacement for libmysql. PHP extensions that use libmysql to connect to MySQL, can be modified to support both libmysql and mysqlnd. This adoption has been finished for ext/mysql and ext/mysqli. Of course, an extension can only use one C-library at a time: either libmysql or mysqlnd. Which one gets used is decided at compile time.
      • Nice diagram.
  • Ubuntu
    • How To Enable/Disable PHP Modules In Ubuntu - TecAdmin - How do I Enable/Disable PHP Modules on Ubuntu 22.04, 20.04 & 18.04 systems? Enable or disable a PHP module for specific PHP versions.
      • phpenmod – Used to enable modules in PHP
      • phpdismod – Used to disable modules in PHP
      • phpquery – Used to view status of modules of PHP
  • ubuntu - php-mysqlnd - Virtual Packages
  • What is the difference between mysqli and mysqlnd php extensions?
    • database - What is the difference between mysqli and mysqlnd php extensions? - Joomla Stack Exchange
      • I'm considering upgrading from PHP7.4 to PHP8.0 on a Joomla 3.10.8 installation. I was surprised to find that my hosting provider only offers mysqlnd as a PHP extension in PHP 8.0, but not mysqli.
      • Postscript: After testing the PHP extensions available, it appears that Joomla will not allow $dbtype to be changed. The solution appears to be to select nd_mysqli extension (as well as mysqlnd) in the PHP8.0 extension list. In configuration.php $dbtype stays as mysqli.
      • The mysqli extension can be compiled against mysqlnd (PHP native driver for MySQL) or libmysql (the shared driver from Oracle). It's a compilation time option. PHP 8.2 will only allow mysqli to be compiled with mysqlnd.
      • The mysqli extension isn't deprecated, but it can be considered obsolete due to the existence of PDO extension. There's no reason to use mysqli if you can use PDO. Joomla allows you to use PDO extension for connecting to the database. PDO_MySQL can also be compiled against libmysql and mysqlnd.
      • Therefore, when setting Joomla, the best setting should be PDO. But the option to use mysqli is still there.
      • When you are using cPanel, the extension page is a non-standard list of PHP extensions. It is confusing because there are two variants for mysqli and PDO, as well as mysqlnd itself. You want to have mysqlnd enabled and the native driver variants of mysqli and PDO, i.e. nd_mysqli and nd_pdo_mysql
  • mysqli
    • There is no extension file for this in the Ubuntu repo.
    • MySQL vs MySQLi in PHP - Tutsplanet - MySQL and MySQli are APIs to connect a MySQL database server in PHP. Each has it’s advantages and disadvantages. Note: By now MySQL extension is deprecated and it is not safe to use in future applications. It will be there for sometime for backward compatibility.
    • MySQLi can no longer be compiled with `libmysqli` - PHP 8.2 • PHP.Watch
      • Since PHP 5.4, mysqlnd is the default library, but it was possible to compile mysqli and pdo_mysql extensions with libmysql with a compile-time configuration flag.
      • Since PHP 8.2 and later, compiling mysqli extension with libmysql is no longer supported.
    • Enable the MySQLi Extension on Linux | Baeldung on Linux
      • Explore the MySQLi extension on a Linux system and learn how to enable it.
      • This article has various grep tests that you can use to verify status.
    • PHP MySQLi Functions | W3Schools
      • For the MySQLi functions to be available, you must compile PHP with support for the MySQLi extension.
      • The MySQLi extension was introduced with PHP version 5.0.0. The MySQL Native Driver was included in PHP version 5.3.0.
    • MySQLi - Wikipedia
  • PDO_MySQL
  • General
    • PHP: Choosing a library - PHP Manual (php.net)
      • The mysqli and PDO_MySQL PHP extensions are lightweight wrappers on top of a C client library. The extensions can either use the mysqlnd library or the libmysqlclient library. Choosing a library is a compile time decision.
      • The mysqlnd library is part of the PHP distribution. It offers features like lazy connections and query caching, features that are not available with libmysqlclient, so using the built-in mysqlnd library is highly recommended. See the mysqlnd documentation for additional details, and a listing of features and functionality that it offers.
  • PDO Tutorials
  • PHP Official Manual
    • PHP: Overview of the MySQL PHP drivers - Manual | php.net
      • There are several PHP APIs for accessing the MySQL database. Users can choose between the mysqli or PDO_MySQL extensions.
      • This guide explains the terminology used to describe each API, information about choosing which API to use, and also information to help choose which MySQL library to use with the API.
    • PHP: Choosing an API - Manual | php.net - PHP offers different APIs to connect to MySQL. Below we show the APIs provided by the mysqli and PDO extensions.
    • PHP: Installation - Manual | php.net - No meta description
      • The common Unix distributions include binary versions of PHP that can be installed. Although these binary versions are typically built with support for the MySQL extensions, the extension libraries themselves may need to be installed using an additional package.
      • The mysqli extension was introduced with PHP version 5.0.0. The MySQL Native Driver was included in PHP version 5.3.0.
      • For example, on Ubuntu the php5-mysql package installs the ext/mysql, ext/mysqli, and pdo_mysql PHP extensions. On CentOS, the php-mysql package also installs these three PHP extensions.
      • On Windows, php_mysqli.dll DLL must be enabled in php.ini.
  • Mysql vs MySQLi vs PDO
    • database - What is the difference between mysqli and mysqlnd php extensions? - Joomla Stack Exchange
      • The mysqli extension can be compiled against mysqlnd (PHP native driver for MySQL) or libmysql (the shared driver from Oracle). It's a compilation time option. PHP 8.2 will only allow mysqli to be compiled with mysqlnd.
      • The mysqli extension isn't deprecated, but it can be considered obsolete due to the existence of PDO extension. There's no reason to use mysqli if you can use PDO. Joomla allows you to use PDO extension for connecting to the database. PDO_MySQL can also be compiled against libmysql and mysqlnd.
      • Therefore, when setting Joomla, the best setting should be PDO. But the option to use mysqli is still there.
      • When you are using cPanel, the extension page is a non-standard list of PHP extensions. It is confusing because there are two variants for mysqli and PDO, as well as mysqlnd itself. You want to have mysqlnd enabled and the native driver variants of mysqli and PDO, i.e. nd_mysqli and nd_pdo_mysql
    • [4.0] Installation DB Order by brianteeman · Pull Request #19780 · joomla/joomla-cms · GitHub
      • Q: I guess PDO is for specific needs and those who know that will choose it.
      • A: No, not really. It's almost the same as MySQL versus MySQLi on PHP 5 (before the former got deprecated). It's just a different underlying API being used to process database transactions. With our abstraction layer the only thing that should matter is whether you have the dependencies for each driver installed. PDO's not some "super advanced special case" system (quite the opposite, most of the PHP ecosystem only uses or supports PDO based database connections, CMS' like Joomla and WordPress are the exception to this pattern where the non-PDO based PHP extensions are used more dominantly).
    • Why You Should Use PDO Instead of MySQLi | Conetix - In this post, I want to look at why you should consider migrating away from MySQLi to PDO in your PHP applications.
    • MySQL vs MySQLi vs PDO Performance Benchmark, Difference and Security Comparison | by Haseeb A. Basil | Medium
    • PDO vs. MySQLi: Which Should You Use? | Envato Tuts+ - When accessing a database in PHP, we have two choices: MySQLi and PDO. So what should you know before choosing one? The differences, database support, stability, and performance concerns will be outlined in this article.
    • PDO vs. MySQLi: The Battle of PHP Database APIs - Long gone are the days of using the mysql_ extension, as its methods have been deprecated since PHP 5.5 and removed as of PHP 7. Alas, the internet is still plagued with a ton of old tutorials that beginners will simply copy/paste and use on a shared hosting platform with an older PHP version, thus continuing its legacy.
    • MySQLi vs PDO vs MySQLnd: PHP Database Extensions Compared - Learn how to choose the best PHP database extension for your web application. Compare MySQLi, PDO, and MySQLnd features, pros, and cons.
    • php - What the different between MySQL Native Driver and MySQL Client Library - Stack Overflow - I want to know the different between MySQL Native Driver and MySQL Client Library and when to use both of them.
  • cPanel PHP Extensions
  • ADOdb vs PDO
    • ADOdb - Database Abstraction Layer for PHP [ADOdb] - ADOdb is a database abstraction layer for PHP. Compatible with most supported databases, it provides an easy way to use the same code against all databases, Portable code can be written that runs unchanged. It also simplifies PHP database access, and includes an extended date/time library, database managed session control, XML database management, and an active record library
    • GitHub - ADOdb/ADOdb: - ADOdb is a PHP database class library that provides powerful abstractions for performing queries and managing databases. ADOdb also hides the differences between DB engines so you can easily switch them without changing your code.
    • PDO (new) [ADOdb]
      • Using ADOdb effectively eliminates the need to use the PHP PDO driver as it hides the PHP level command:
Published in PHP
Read more...
Saturday, 27 January 2024 11:36

Veeam backup fail after Windows Update

Issue

My local backup is using 'Veeam Endpoint for Windows' and an external USB Hard Drive which were working fine until after a Windows Update. It took a while before I worked out the link between the update and Veeam.

I would get errors like these shown below.

Error: Full backup file merge failed

26/01/2024 18:09:41 :: Full backup file merge failed Error: Agent: Failed to process method {Transform.Patch}: The media is write protected.

Error: Asynchronous read operation failed

26012024 215538  Error The device is not ready. Asynchronous read operation failed Failed to upload disk. Agent failed to process method {DataTransfer.SyncDisk}. Exception from server The device is not ready. Unable to retrieve next block transmission command. Number of already processed blocks [86184]. Failed to download disk 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'

Cause

After a Windows Update, there was something wrong with the mount point for the 'Recovery Partition' because the update had corrupted the partition somehow. Because of this corruption, Veeam cannot correctly mount the 'Recovery Partition' so the backup fails.

Solution

Mount and Unmount the 'Recovery Partition' (Add and Remove a drive letter)

  • What this seems to do is fix the mount point for the partition and this allows Veeam to complete it's backup.
  • When I added a drive letter I found there were no files on my 'Recovery Partition'.
  • There are many different ways you can do this but here are a few:
    • diskpart (via cmd)
    • Powershell
    • Install a partition manager into Windows.
    • Boot from a partition manager, assign a drive letter, load Windows, boot from the partition manager again and remove the drive letter.

The backup will show a success message as follows:

Research

Published in Applications
Read more...
Page 3 of 95