UP | HOME
Ravi Sagar | Home | Blog

Mastering Drupal

Table of Contents

1 Setup LAMP on Arch for Drupal development on local computer

The following setup is for local Drupal development. Source: https://wiki.archlinux.org/title/Apache_HTTP_Server

1.1 Install Apache and start it

sudo pacman -S apache
systemctl start httpd.service

Main configuration is /etc/httpd/conf/httpd.conf and default DocumentRoot is /srv/http

Check apache configurations using apachectl configtest command.

1.2 Create user home directory (optional)

mkdir ~/public_html
touch ~/public_html/index.html
echo "Ravi's home" >> ~/public_html/index.html
chmod o+x ~
chmod o+x ~/public_html
chmod -R o+r ~/public_html
systemctl restart httpd.service

Now you can access your personal home page at localhost/~ravisagar

1.3 TLS

In /etc/httpd/conf/httpd.conf, uncomment the following three lines:

LoadModule ssl_module modules/mod_ssl.so
LoadModule socache_shmcb_module modules/mod_socache_shmcb.so
Include conf/extra/httpd-ssl.conf

If using Certbot (certbot –apache), the following line needs to be uncommented as well:

LoadModule rewrite_module modules/mod_rewrite.so

1.4 Virtual hosts

If you want to have more than one host, uncomment the following line in /etc/httpd/conf/httpd.conf

Include conf/extra/httpd-vhosts.conf

Content of the conf/extra/httpd-vhosts.conf If the setup if for local computer then ignore the ssl part.

<VirtualHost *:80>
    ServerAdmin admin@sparxsys.com
    ServerName sparxsys.local
    ServerAlias www.sparxsys.local
    DocumentRoot "/home/ravisagar/projects/sparxsys.local/public_html/web"
    ErrorLog "/home/ravisagar/projects/sparxsys.local/logs/error.log"
    CustomLog "/home/ravisagar/projects/sparxsys.local/logs/access.log" common
    <Directory "/home/ravisagar/projects/sparxsys.local/public_html/web">
        Require all granted
        AllowOverride All
    </Directory>
    #RewriteEngine on
    #RewriteCond %{SERVER_NAME} =indiaunveiled.in [OR]
    #RewriteCond %{SERVER_NAME} =www.indiaunveiled.in
    #RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

<VirtualHost *:443>
    ServerAdmin admin@sparxsys.com
    ServerName sparxsys.local
    ServerAlias www.sparxsys.local
    DocumentRoot "/home/ravisagar/projects/sparxsys.local/public_html/web"
    ErrorLog "/home/ravisagar/projects/sparxsys.local/logs/error.log"
    CustomLog "/home/ravisagar/projects/sparxsys.local/logs/access.log" common
    <Directory "/home/ravisagar/projects/sparxsys.local/public_html/web">
        Require all granted
    </Directory>
    #RewriteEngine on
    #RewriteCond %{SERVER_NAME} =staging.ravisagar.in [OR]
    #RewriteCond %{SERVER_NAME} =www.staging.ravisagar.in
    #RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
    SSLEngine on
    Include /etc/letsencrypt/options-ssl-apache.conf
    SSLCertificateFile /etc/letsencrypt/live/www.staging.ravisagar.in/cert.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/www.staging.ravisagar.in/privkey.pem
    SSLCertificateChainFile /etc/letsencrypt/live/www.staging.ravisagar.in/chain.pem
</VirtualHost>

1.5 Modify your /etc/hosts and add your virtual hosts

127.0.0.1 sparxsys.local
127.0.01 www.sparxsys.local

That is it, now you can open http://sparxsys.local/ in your browser

1.6 Install php

Just install php7 to be on the safer side.

sudo pacman -S php7 php7-apache php7-gd

In /etc/httpd/conf/httpd.conf, comment the line:

#LoadModule mpm_event_module modules/mod_mpm_event.so

and uncomment the line:

LoadModule mpm_prefork_module modules/mod_mpm_prefork.so

Place this at the end of the LoadModule list:

LoadModule php7_module modules/libphp7.so
AddHandler php-script .php

Place this at the end of the Include list:

Include conf/extra/php7_module.conf

Check your php.ini file in use using php --ini Modify your /etc/php7/php.ini and uncomment the following line.

extension=gd
extension=iconv
extension=pdo_mysql
zend_extension=opcache
opcache.enable_cli=1
opcache.memory_consumption=128
opcache.interned_strings_buffer=8
opcache.max_accelerated_files=4000
opcache.revalidate_freq=60

upload_max_filesize = 10M
post_max_size = 10M

Switch to php7 (if needed, didn't work properly though)

sudo rm /usr/bin/php
sudo ln -s /usr/bin/php7 /usr/bin/php
sudo rm /usr/bin/php-config
sudo ln -s /usr/bin/php-config7 /usr/bin/php-config
sudo rm /usr/bin/phpize
sudo ln -s /usr/bin/phpize7 /usr/bin/phpize
php -v

1.7 Install mysql

sudo pacman -S mariadb

If you are not able to start mariadb then perform these steps.

systemctl stop mariadb
sudo rm -R /var/lib/mysql/*
sudo mysql_install_db --user=mysql --basedir=/usr --datadir=/var/lib/mysql
sudo systemctl start mariadb

Setup mysql

sudo mysql_secure_installation

Create database

CREATE DATABASE db_ CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

1.8 Test PHP works

echo "<?php phpinfo(); ?>" > test.php

Now go to http://localhost/~ravisagar/test.php and check whether you see php version.

2 Install Composer

Follow the instructions here: https://getcomposer.org/download/ You may want to install it in ~/Downloads. After following the instructions you will get composer.phar which you can move to your /usr/local/bin.

sudo mv ~/Downloads/composer /usr/local/bin/

3 Move existing site to drupal-composer/drupal-project

Update: The following steps didn't work because I got error related to old modules. So I decided to install fresh Drupal.

You can move your existing site that you made without composer to drupal-composer.

3.1 Install new drupal site

composer create-project drupal-composer/drupal-project:9.x-dev public_html --no-interaction

Create config/sync directory in composer root, inside publichtml.

3.2 Copy all custom themes, modules, files, and settings to the new project

Copy files

cp -a ~/webfaction-sites/sparxsys/var/www/html/sparxsys.live/public_html/web/sites/default/files/. ~/projects/sparxsys.local/public_html/web/sites/default/files/
cp ~/webfaction-sites/sparxsys/var/www/html/sparxsys.live/public_html/web/sites/default/settings.php ~/projects/sparxsys.local/public_html/web/sites/default/

Copy settings.php and make sure database settings are correct.

cp ~/webfaction-sites/sparxsys/var/www/html/sparxsys.live/public_html/web/sites/default/settings.php ~/projects/sparxsys.local/public_html/web/sites/default/

Restore your database

mysql -u root -p db_local_sparxsys < ~/webfaction-sites/sparxsys/home/drupal/backup/sparxsys.com/2021-12-17_db.sql

Install modules needed. Check in your existing Drupal installation. Install all the modules previously installed.

composer require drupal/metatag

Install theme.

composer require drupal/bootstrap

Create directory for custom theme.

mkdir web/themes/custom

Move old subtheme to this directory.

cp -r ~/webfaction-sites/sparxsys/var/www/html/sparxsys.live/public_html/web/themes/ravisagartheme ~/projects/sparxsys.local/public_html/web/themes/custom/

Add the following in your ravisagartheme/ravisagartheme.info.yml

core_version_requirement: ^8 || ^9

Update db and clear cache

vendor/bin/drush updb
vendor/bin/drush cache-rebuild

3.3 Change site url for drush

Check current Drush configs path and in the /public_html/vendor/drush/drush/drush.yml file add the following.

options:
  uri: 'http://sparxsys.local'

4 Correct file permissions

chmod 755 /drupalroot Go inside drupal directory

find . -type d -exec sudo chmod 755 '{}' \; find . -type f -exec sudo chmod 644 '{}' \;

sudo chmod 777 -R sites/default/files/ sudo chmod 444 sites/default/settings.php

5 Most common modules needed on a new site

Very important modules

Pathauto

Metatag

Geshifilter

Setup site

Create a user other than admin

Create Blog

Enable media and media library

Create Topics, add to the node

Add image field to Blog

Setup pathauto

Setup Metatag

Setup Google Analytics

6 Drupal errors and issues

6.1 index.php in the url

Install redirect module and make sure the "Enforce clean and canonical URLs." is checked

6.2 How To Solve Your lock file does not contain a compatible set of packages Please run composer update Error?

composer install --ignore-platform-reqs

6.3 Composer memory errors

Run the command prefixing with giving unlimited memory to composer.

COMPOSER_MEMORY_LIMIT=-1 composer remove drupal-composer/drupal-scaffold

6.4 Scaffold error

https://patrickmichael.co.za/drupal-88-9-upgrade-8919-926-tutorial

sudo composer selfupdate --1
composer remove drupal-composer/drupal-scaffold
sudo composer selfupdate --2
composer require drupal-composer/drupal-scaffold

7 Update Drupal using composer with a proper development workflow

Keeping your Drupal site updated is important. Doing it with Composer is easy but it has to be done properly. In this blog let me share the steps I follow to update Drupal using a proper development workflow. These are the steps. Enjoy.

7.1 Recommended setup

We all have different ways of working. This setup works really well for me because I am the only developer working on this site.

  • Local site: www.ravisagar.test

This is for significant development work like adding a new module, design changes or some major functionality.

  • Staging site: www.staging.ravisagar.in

Only for testing.

  • Live site: www.ravisagar.in

Always has the most recent content like new blogs, new images. Sometimes I do some small css changes directly on live.

7.2 Upgrade a Drupal site

These are the steps involed in upgrading a Drupal site.

7.2.1 Refresh local with production data and configurations

I do lot of small changes and tweaks directly on production and of course production has latest content as well. So from time to time I refresh my local with production but I also do this when I have to upgrade a site.

  1. Take mysqldump of production database and save it in files/database/YYYY-MM-DD.tar.gz
  2. Git push (it won't have files directory). Do this if you installed new module directly on production or done some changes to the theme css.
  3. Go to local and rsync files directory (that has images and latest database). Change the path accordingly.
rsync --dry-run -av user@production:/PATH_TO_LIVE_SITE/sites/default/files/ files/
rsync -av drupal@ravisagar.in:/var/www/html/ravisagar.live/public_html/web/sites/default/settings.php settings.php
  1. Now local has files/ directory from live.
  2. Restore database. Now local has live database.
  3. Perform git pull and it will have any modules or theme changes done on production.
  4. Run composer install (if you installed any module on production). I always run this. Just in case.
  5. Clear cache vendor/bin/drupal cache:rebuild all
  6. Now you have local running as same configuration and content from production.

7.2.2 Update Drupal on local

First check this page for up to date information. https://www.drupal.org/docs/updating-drupal/updating-drupal-core-via-composer

Match composer version

sudo composer selfupdate --1

Put site in maitenance mode and clear cache.

vendor/bin/drush state:set system.maintenance_mode 1
vendor/bin/drush cache:rebuild

Update modules using composer to make them compatible with Drupal 9. Check modules that needs an update.

composer outdated "drupal/*"

For minor version upgrade do this for each module.

COMPOSER_MEMORY_LIMIT=-1 composer update drupal/address --with-dependencies
COMPOSER_MEMORY_LIMIT=-1 composer update drupal/adstxt --with-dependencies
COMPOSER_MEMORY_LIMIT=-1 composer update drupal/bootstrap --with-dependencies
COMPOSER_MEMORY_LIMIT=-1 composer update drupal/console --with-dependencies
COMPOSER_MEMORY_LIMIT=-1 composer update drupal/ctools --with-dependencies
COMPOSER_MEMORY_LIMIT=-1 composer update drupal/easy_install --with-dependencies
COMPOSER_MEMORY_LIMIT=-1 composer update drupal/geofield drupal/geofield_map drupal/geshifilter --with-dependencies
COMPOSER_MEMORY_LIMIT=-1 composer update drupal/h5p drupal/indieweb drupal/metatag drupal/metatag_views drupal/pathologic drupal/protected_pages drupal/redirect drupal/restui drupal/scheduler drupal/smtp drupal/token drupal/views_slideshow drupal/xmlrpc drupal/xmlsitemap --with-dependencies

For major version upgrade do this for each module.

COMPOSER_MEMORY_LIMIT=-1 composer require drupal/checklistapi:^2.0.0
COMPOSER_MEMORY_LIMIT=-1 composer require drupal/devel:^4.1.3
COMPOSER_MEMORY_LIMIT=-1 composer require drupal/devel_entity_updates:^3.0.1
COMPOSER_MEMORY_LIMIT=-1 composer require drupal/exif:^2.2.0
COMPOSER_MEMORY_LIMIT=-1 composer require drupal/field_token_value:^2.0.1
COMPOSER_MEMORY_LIMIT=-1 composer require drupal/geolocation:^3.7.0
COMPOSER_MEMORY_LIMIT=-1 composer require drupal/honeypot:^2.0.2
COMPOSER_MEMORY_LIMIT=-1 composer require drupal/oembed_providers:^2.0.0
drush updb

Verify if the project uses drupal/core-recommended or drupal/core. Run composer show drupal/core-recommended If drupal/core-recommended is installed, this command returns information about the package. If drupal/core-recommended is not installed, this command returns "Package drupal/core-recommended not found".

If you are running on Drupal 8.8.5 and above then this should work.

composer update drupal/core "drupal/core-*" --with-all-dependencies
drush updatedb
vendor/bin/drupal cache:rebuild all

If the above command doesn't work then find what is blocking it.

COMPOSER_MEMORY_LIMIT=-1 composer prohibits drupal/core 9.3.0

Update Composer back to 2

sudo composer selfupdate --2

In composer.json Remove all lines drupal/core*** except drupal/core-recommended and drupal/core-composer-scaffold. Those two will require all other stuff, the superfluent drupal/core*** just make future upgrading harder.

For upgrading to Drupal 9. Source: https://github.com/drupal-composer/drupal-project/issues/565

COMPOSER_MEMORY_LIMIT=-1 composer require drupal/core-recommended:^9.3.0 drupal/core-composer-scaffold:^9.3.0 drupal/core:^9.3.0  drush/drush:^10.6 symfony/yaml:^4.4.34 drupal/views_responsive_grid:^1.x.dev --update-with-dependencies --no-update
composer update
drush updatedb
vendor/bin/drupal cache:rebuild all

Install these modules post upgrade https://www.drupal.org/project/h5p/releases/2.0.0-alpha1

composer require 'drupal/h5p:2.0@alpha'

https://www.drupal.org/project/geshifilter/releases/8.x-2.0-beta1 composer require 'drupal/geshifilter:2.0@beta'

composer remove drupal/geocoder composer require drupal/geocoder composer require geocoder-php/google-maps-provider COMPOSERMEMORYLIMIT=-1 composer require drupal/googleanalytics:4.0.0

Keep on trying the above command until all the missing modules are installed. Remove them from composer.json.

Also refer to this page for up to date information on how to update Drupal using Composer.

7.2.3 Config export

If you did configuration changes on local site then you need to export them and import them later on production.

vendor/bin/drupal config:export

You can also do it from the UI. Later use the same command to import.

vendor/bin/drupal config:import

Personally I haven't used it yet, because I hardly add any configuration on local and then push to production. I also refresh local database from live from time to time.

7.2.4 Commit your updated Drupal site to the repository

Now you have local running on updated Drupal site with latest content and files from live. Celebrate!

git add .
git commit -m "updating to Drupal 9.3.0"
git push

7.2.5 Test your newly updated site on staging

Composer update on production. I didn't need to update composer on production, I guess because on staging/production I am only going to run composer install. BTW on production ideally run ~ composer install –no-dev~ so that it won't install dev modules on production, but not me, because I like living on the edge.

Time to test your site on staging. Good opportunity to make sure it works on production environment.

  • Go to your staging directory and bring the files from the repository by doing git pull
  • Also bring the new files from production site.
rsync --dry-run -avv user@production:/PATH_TO_LIVE_SITE/sites/default/files/ /PATH_TO_STAGING_SITE/sites/default/files/
  • From the files/database directory restore the mysql database on staging
  • At this staging you have latest files from production, latest database from production and latest upgrade code from local which usually in the composer.json and composer.lock files. The last thing you need to do on staging is to use composer to bring new files need for the update.
  • Add the following line in settings.php and comment the previous array for sync.
sudo chmod 777 web/sites/default/settings.php
$settings["config_sync_directory"] = '../config/sync';
//$config_directories['sync'] = '../config/sync';
sudo chmod 444 web/sites/default/settings.php
composer install
drush updb
vendor/bin/drupal cache:rebuild all

Now your staging is updated.

7.2.6 Update your live.

Ready to update live? Lets do it.

git pull
composer install
drush updatedb
vendor/bin/drupal cache:rebuild all

Simple right? Your live already has the latest database and files. We just need to bring the latest updated composer.json and composer.lock, along with any other changes to theme that you may have done on local.

Now time to open a beer and celebrate.

7.3 Letsencrypt certificate

Just follow this tutorial and you are done: https://linuxhostsupport.com/blog/how-to-install-lets-encrypt-on-centos-7-with-apache/

yum install certbot 
yum -y install certbot-apache

Install certificate using the command.

sudo certbot --apache

It will make changes to the following files for you.

sudo emacs /etc/httpd/conf.d/vhost-le-ssl.conf

It will also add redirect (if you choose) in the following file.

/etc/httpd/conf.d/vhost.conf