You are here:Home»KB»Web Design»CMS»Joomla»Extension Development»How to extract installed extensions from Joomla
Friday, 02 October 2020 14:20

How to extract installed extensions from Joomla

Written by

I found the need to extract extensions from Joomla and these are my notes that I built up while doing that. You can also extract core extensions aswell with a little more effort with some compromises. The reasons for extraction:

  • Build a reference extension for future projects.
  • You no longer have the installation sources but need a copy of the extension.

It is far easier to use com_contacts for a reference extension if you are going to use one from the core. com_content is spread across Joomla becasue it is an integral part rather than a seperate component, this maybe better in joomla 4.

Some Notes

  • This will use com_content as an example which use en-GB
  • I am using Joomla 3.9.21 (using Joomla_3.9.21-Stable-Full_Package.zip)
  • I am only doing MySQL
  • The manifest file is in /joomla/administrator/components/com_content/content.xml
  • The manifest file tells you where all of the files are.
  • [zip] = the new extension location
  • [joomla] = Joomla Full installation zip package (makes surte files a virgin)
  • adding license and readme are optional and can be done as part of your own project
  • i will write an extension extractor (extracting the SQL data from the database might need to be done manually)
  • com_content and com_contact have
  • com_content (j3) is missing:
    • SQL files
    • SQL Manifest reference
    • Menu definitions in the manifiest
  • com_contact (j3) has everything present like a normal component
  • Use com_contact is a better extension to use for a reference component as not using dynamic menus like com_content and its manifest is complete.
  • Most core components have a reduced manfest and will need manual correction.
  • com_content admin menu items are programatically controlled and are set by presets in these files:
    • [joomla]/administrator/components/com_menus/presets/joomla.xml
    • [joomla]/administrator/components/com_menus/presets/modern.xml
  • About `Fields` and `Field Groups` menu items:
    • In com_content when you disable custom fields in `Articles --> Options --> Integration --> Select Yes in "Enable Custom Fields" option.`, these menus get disabled in both the main menu and the side menu of the component. the same also happens if you disable the com_fields component.
    • JDownloads has these menus and when i disable com_fields they disappear from the components side menu but not the JDownloads component menu. When i click on them while com_fields is off I get an error becasue the link no longer works correctly. This means that the componnent menus defined in the manifest are not dynamic but the ones defined in the presets (joomla.xml and modern.xml) are.
    • com_content side menus are controlled here: 
      [joomla]/administrator/components/com_content/helpers/content.php -->function addSubmenu()

       

  • You can only have 1 level of <submenu> when adding via manifest i.w. `Components --> QWDemoBar --> Products' (`Components --> <menu> --> <submenu>`)

Instructions for component extraction

These will work for Joomla core (with some issues) and 3rd party components with no issues.

  1. Create a target folder structure for extension
    [zip]/com_content/
[zip]/com_content/admin/
[zip]/com_content/admin/language/
[zip]/com_content/admin/language/en-GB/
[zip]/com_content/media/ 
[zip]/com_content/site/
[zip]/com_content/site/language/
[zip]/com_content/site/language/en-GB/
  2. Grab Files
    • copy [joomla]/administrator/language/en-GB/en-GB.com_content.ini to [zip]/com_content/admin/language/
    • copy [joomla]/administrator/language/en-GB/en-GB.com_content.sys.ini to [zip]/com_content/admin/language/
    • copy [joomla]/administrator/components/com_content/ to [zip]/com_content/admin/
    • copy [joomla]/media/com_content/ to [zip]/com_content/media/
    • copy [joomla]/language/en-GB/en-GB.com_content.ini to [zip]/com_content/site/language/
    • copy [joomla]/components/com_content/ to [zip]/com_content/site/
  3. Correct File Structure
    • move [zip]/com_content/admin/content.xml to [zip]/com_content/
  4. Add Database Files and manifest references if missing (This section applies only to some core components where they are missing)
    • create [zip]/com_content/admin/sql/
    • create [zip]/com_content/admin/sql/install.mysql.utf8.sql
    • create [zip]/com_content/admin/sql/uninstall.mysql.utf8.sql
    • create [zip]/com_content/admin/sql/updates/
    • create [zip]/com_content/admin/sql/1.0.0.sql with the content: 
      # Placeholder file for database changes for version 1.0.0
    • extract the installation SQL from [joomla]/installation/sql/mysql/joomla.sql
      • You will have to make a best guess about which SQL code to get and looking at the database with phpmyadmin will help.
      • Search for all #__content and this should show you all of the required tables, this should go in the install.mysql.utf8.sql
      • com_content the code you need is at lines 318 - 449
    • In the uninstall.mysql.utf8.sql add a drop statement for each of the tables you have just added to the install SQL code. (i.e. DROP TABLE IF EXISTS `#__content_categories`; )
    • Add the SQL references into the manifest file for install/uninstall
      [zip]/com_content/admin/content.xml
  5. Add Menu Links to the manifest (This section applies only to some core components where they are missing)
    • If you are extracting com_content add the following code immdiately after the <administration> tag or look at com_contact manifest file for examples of how to set your menu out:

      NB: the menu links below do not have index.php
      <menu link="option=com_com_content">MOD_MENU_COM_CONTENT</menu>
<submenu>
    <menu link="option=com_content">MOD_MENU_COM_CONTENT_ARTICLE_MANAGER</menu>
    <menu link="option=com_categories&amp;extension=com_content">MOD_MENU_COM_CONTENT_CATEGORY_MANAGER</menu>            
    <menu link="option=com_content&amp;view=featured">MOD_MENU_COM_CONTENT_FEATURED</menu>
    <menu link="option=com_fields&amp;context=com_content.article">MOD_MENU_FIELDS</menu>
    <menu link="option=com_fields&amp;view=groups&amp;context=com_content.article">MOD_MENU_FIELDS_GROUP</menu>            
</submenu>
    • Notes
      • `Add New Article` and `Add New Category` submenus cannot be re-created via the manifest menu creation. These are only shortcuts anyway so no functionality is lost.
      • The new <menu> and first <submenu> are the same. In the MOD_MENU the primary menu item is not a link
      • if you want to convert to a full separate extension you need to take the translations from the joomla.xml and add them in to your extension natively
  6. Correct Manifest File (You dont have to move the file for the installer to work - i dont thing, but you should for correctness)
    • After moving the manifest file you have to modify it to match new file/folder structure, in particular,
    • The manifest file itself does not need to be added in the <file> copy list because it will be copied automatically so remove it if the reference is present.
    • For com_content in <files folder="admin"> I removed <folder>elements</folder> because there was no matching folder in the admin filesystem. This must be a Joomla bug.
    • The method I would use is
      • Just open up for new component folder
      • Go into the site folder
      • and one by one make sure that the relevant commands to copy the files and folders in the site folder are present in the manifest.
      • Remember language files are dealt with seperately but the sql folder in admin must be copied.
      • Once done do the admin fodler
      • Lastly check the media folder reference
  7. Backup
    • You should backup your work before continuing
    • An example filename is: com_content - Extracted from J3.9.21 (All Native).zip
  8. Convert to normal extension (Some remedial work to finish up)
    • if you have extracted com_content you will need to rename the menu translations in the manifest as outlined below 
      MOD_MENU_COM_CONTENT --> COM_CONTENT : Ignore this line as COM_CONTENT already exists
MOD_MENU_COM_CONTENT_ARTICLE_MANAGER --> COM_CONTENT_ARTICLE_MANAGER
MOD_MENU_COM_CONTENT_CATEGORY_MANAGER --> COM_CONTENT_CATEGORY_MANAGER
MOD_MENU_COM_CONTENT_FEATURED --> COM_CONTENT_FEATURED_ARTICLES
MOD_MENU_FIELDS --> COM_CONTENT_FIELDS
MOD_MENU_FIELDS_GROUP --> COM_CONTENT_FIELD_GROUPS
      and then add the same translations as a block into the admin translations file [zip]/admin/language/en-GB/en-GB.com_content.sys.ini as show below. 
      ;Menu (previously handled by com_admin presets)
COM_CONTENT_ARTICLE_MANAGER="Articles"
COM_CONTENT_CATEGORY_MANAGER="Categories"
COM_CONTENT_FEATURED_ARTICLES="Featured Articles"
COM_CONTENT_FIELDS="Fields"
COM_CONTENT_FIELD_GROUPS="Field Groups"
      This is required to make the component a standalone verion of the core component.
    • (optional) add all missing information as per my boilerplate i.e. update server - The current manifest file will work but is not 100% complete
    • upgrade manifest file to my format (see boiler plate)
    • Add the following missing translations to admin/languages/en-GB/eb-GB.com_content.sys.ini 
      ;install/update/uninstall system
COM_CONTENT_NAME="Content (Component)" ;This might not be needed
COM_CONTENT_DESCRIPTION="A standalone verion of the Joomla core component, content."
    • (optional)(recommended) Install/Update/Uninstall Script File
      • Add manifest reference. This will also copy it into the admin section when installing the component or the root folder of the extention if not a component.
      • create file [zip]/com_content/script.php (core apps dont have this as it is not required for them) - this is just to make it like a normal extension 
        <!-- Script: Install, Update, Uninstall -->
<scriptfile>script.php</scriptfile>
      • add the following translations to admin/languages/en-GB/eb-GB.com_content.sys.ini 
        ; script.php (install/update/uninstall)
COM_CONTENT_INSTALL_TEXT="The component has been installed."
COM_CONTENT_UNINSTALL_TEXT="The component has been uninstalled."
COM_CONTENT_UPDATE_TEXT="The component has now been updated to version %s."

; script.php - Before Actions
COM_CONTENT_PREFLIGHT_DISCOVER_INSTALL_TEXT="Content preflight discover install script."
COM_CONTENT_PREFLIGHT_INSTALL_TEXT="Content preflight install script."
COM_CONTENT_PREFLIGHT_UNINSTALL_TEXT="Content preflight uninstall script."
COM_CONTENT_PREFLIGHT_UPDATE_TEXT="Content preflight update script."

; script.php - After Actions
COM_CONTENT_POSTFLIGHT_DISCOVER_INSTALL_TEXT="Content postflight discover install script."
COM_CONTENT_POSTFLIGHT_INSTALL_TEXT="Content postflight install script."
COM_CONTENT_POSTFLIGHT_UNINSTALL_TEXT="Content postflight uninstall script."
COM_CONTENT_POSTFLIGHT_UPDATE_TEXT="Content postflight update script."
    • (optional) add dummy css file
    • (optional) grab corresponding modules and plugins and make a package
    • zip/compress contents of [zip]/com_content/ to com_content - Extracted from J3.9.21 (Native Modified).zip

Renaming Extension / Refactoring Component

You cannot install the extension you have just made because it will break your joomla installation, so you must refector it to a new name such as com_qwhelloworld.

Make sure you have made another backup before proceededing example file name = com_content - Extracted from J3.9.21 (Native with manifest, script and translations upgraded).zip

Again these instructions will outlione how to refactor com_content to com_qwhelloworld.

I did an article Rename a Joomla plugin or create a second instance of it that might be of use.

Rename the following files:

You can also search for `content` with your favorite file search tool.

  • [zip]/content.xml --> qwhelloworld.xml + manifest reference
  • [zip]/admin/content.php --> qwhelloworld.php + manifest reference
  • [zip]/admin/en-GB/en-GB.com_content.sys --> en-GB.com_qwhelloworld.sys + manifest reference
  • [zip]/admin/en-GB/en-GB.com_content.ini -> en-GB.com_qwhelloworld.ini + manifest reference
  • [zip]/admin/helpers/content.php --> qwhelloworld.php
  • [zip]/admin/helpers/html/contentadministrator.php --> qwhelloworldadministrator.php
  • [zip]/site/content.php --> qwhelloworld.php + manifest reference
  • [zip]/site/en-GB/en-GB.com_content.ini -> en-GB.com_qwhelloworld.ini

Text Replacing

You have 2 ways of perfoming text replacing in your extension:

  • Use an IDE of your choice (Netbeans etc..)
  • Open all files with Notepad++

What i did

I did not just rename `Content -> QWHelloWorld`, because of the name of the component (com_content) there are a lot of similiar name functions using the word Content and those should not be changed.

Normally if you are doing a uniquely name extension this should not be an issue i.e. com_contact.

  1. com_contact --> com_qwhelloworld
  2. COM_CONTACT --> COM_QWHELLOWORLD
  3. Contact --> Qwhelloworld (Just ignore translations)

So I worked through the list below until I got too `Content -> QWHelloWorld`, again used the Find and replace option in Netbeans but this time I inspected every line/match that was found for issues and made notes of these, then now i performed `Content -> QWHelloWorld`so when the unwanted changed were made I could go through and correct only the errors becasue I had built up a list of changes to revert, This is how I got my corrections section.

Perform the following text renames

Now perform the following renames in the order they appear and they are case sensitive

  • Rename Extension
    • COM_QWHELLOWORLD="Articles" --> COM_QWHELLOWORLD="Qwhelloworld (Component)"
  • Database
    • #__content --> #__com_qwhelloworld
  • Functions
    • function content --> function qwhelloworld
  • Classes
    • JHelperQwhelloworld --> JHelperContent
  • File Reference
    • content.php --> qwhelloworld.php
  • Misc
    • contentadministrator --> qwhelloworldadministrator ??? not sure about this one, this might be a native joomla thing
  • General
    • com_content --> com_qwhelloworld
    • COM_CONTENT --> COM_QWHELLOWORLD
    • Content -> Qwhelloworld

Corrections

Now because of the name com_content which gives Content which in turn is not unique, you will find you need to do some corrections which I have outlined below. Please be aware this is not exhaustive as I might of missed stuff

  • Event Statements
    • onQWHelloWorldAfterSave --> onContentAfterSave
    • onQWHelloWorldPrepare --> onContentPrepare
    • onQWHelloWorldAfterTitle --> onContentAfterTitle
    • onQWHelloWorldBeforeDisplay --> onContentBeforeDisplay
    • onQWHelloWorldAfterDisplay --> onContentAfterDisplay 
    • beforeDisplayQWHelloWorld --> beforeDisplayContent
    • afterDisplayQWHelloWorld --> afterDisplayContent
    • Joomla 4 only
      • onQWHelloWorldBeforeChangeFeatured --> onContentBeforeChangeFeatured
      • onQWHelloWorldAfterChangeFeatured --> onContentAfterChangeFeatured
  • Functions
    • getQwhelloworldLanguages( --> getContentLanguages(
    • getQuickiconQwhelloworld( --> getQuickiconContent(
    • _buildQwhelloworldOrderBy( --> _buildContentOrderBy( ??? not sure about this one, maybe it needs leaving
  • Variable Names
  • JS
    • DOMQWHelloWorld --> DOMContent
  • Headers
  • Code Comments
    • // QWHelloWorld is generated  --> // Content is generated
    • Joomla 4 only
      • Joomla! QWHelloWorld Management System --> Joomla! Content Management System
  • DB Data
    • "type":"QWHelloWorld" --> "type":"Content" ??? not sure about this one
  • Translations
    • COM_QWHELLOWORLD_ARTICLE_CONTENT="QWHelloWorld" --> COM_QWHELLOWORLD_ARTICLE_CONTENT="Content"
    • QWHelloWorld Item Associations --> Content Item Associations
    • Table of QWHelloWorld --> Table of Contents
    • QWHelloWorld Settings -> Content Settings ??? not corrected this as I am not sure it needs changing
    • QWHelloWorld Dashboard --> Content Dashboard ??? not corrected this as I am not sure it needs changing
    • QWHelloWorld Filter Search --> Content Filter Search ??? not corrected this as I am not sure it needs changing

Corrections - com_content Table Management (JTable)

Because com_content is blended into the core we need to correct a few things so the correct table is found and used. This section should only be neded for com_content but this procedure can be adapted if needed.

Consider this code:

// I think calls the table instance for the component
$contentTable = JTable::getInstance('Qwhelloworld', 'JTable');

// controlling the Feature artciles which will help you work out what to change
// [zip]/admin/models/article.php
$table = $this->getTable('Featured', 'QwhelloworldTable');

// [zip]/admin/models/feature.php
public function getTable($type = 'Featured', $prefix = 'QwhelloworldTable', $config = array())

Instructions

  • Create missing JTable file
    • copy [zip]/admin/tables/featured.php --> [zip]/admin/tables/content.php
    • rename Featured to Content
    • alter '#__com_qwhelloworld_frontpage' --> '#__com_qwhelloworld'
    • alter 'content_id' is renamed to 'id'
    • in constructor rename  alter com_content --> com_qwhelloworld
  • examine [joomla]/libraries/src/Table/Content.php and you will see the construct class for Content
    • i do not know if i need all of this file, all of what is in the contructor here.
    • I will use all of what is in the constructor modified to the database style as shown in featured. i can always alter the remaing files ie.e com_conent to com_qwhelloworld later
    • the main thing to take from here is to see that 'content_id' is renamed to 'id'
  • We now have our table file created so now need to correct the JTable references
    NB: This is where I got upto and not further
    • These are examples of what you can try. Rememebr it is 'Qwhelloworld' that is the last text change to fix. 
      JTable::getInstance('Qwhelloworld', 'JTable'); --> JTable::getInstance('QwhelloworldContent', 'JTable')
getTable($type = 'Qwhelloworld', --> getTable($type = 'QwhelloworldContent',
    • https://docs.joomla.org/Using_the_JTable_class
    • https://docs.joomla.org/Creating_content_using_JTableContent
    • I dont think the class should be prefixed with JTable like in the core
    • I think 'Qwhelloworld' for JTable should be converted to 'QwhelloworldContent'

Install the Extension

Once you have done these changed you can see if you extension works and make any corrections as required. If there any unforseen issues try an fix them or you can just start again with your backup.

Most erros casued during the installation are probably caused by the manifest or files with the wrong names.

Joomla Debug is your friend

View the extension in site and admin for different errors to those during installation but this time with Joomla Debug on and you can then go through and fix the errors. I would also recommend making notes.

Make it reference extension (com_content)

This is for me really.

  • Add Files (referenced as needed in the manifest)
    • CSS file
    • CHANGELOG.md
    • LICENSE
    • README.md

Plugin, Module and Template Extraction

This is far simpler, just go to the relevant folder and zip the contents. You have now extracted your choosen extension.

Here are some example locations:

  • Plugin: [joomla]/plugins/content/qwhelloworld/
  • Module: [joomla]/modules/mod_qwhelloworld/
  • Template: [joomla]/templates/qwhelloworld/

 

Read 531 times Last modified on Monday, 05 October 2020 11:56
Published in Extension Development
back to top