Sugar Dockerized (Sugar application not included. Download at www.sugarcrm.com). For SugarCRM services, reach out to https://www.naonis.tech/services/sugar-crm.html
This repository will help you deploy a Docker based development only full stack for Sugar, meeting all the platform requirements for a different set of platform combinations.
If you find this software useful, please consider supporting the work that went into it, with a monthly amount. Thank you!
There are few stacks available, with in itself multiple platform combinations. You can read more about the specific stacks on the links below:
You will find additional stacks within the stack directory of the project.
For most stacks, there are both the pre-built version (eg on Sugar 9: ./stacks/sugar9/php73.yml
) and a locally built version (eg on Sugar 9: ./stacks/sugar9/php73-local-build.yml
). The locally built version will be built run-time, and therefore those stacks will let you specify additional changes you might require to the docker images provided. Local builds will take much longer to deploy than pre-built ones.
There are mainly three types of stack:
There are multiple stack components as docker containers, that perform different duties. Not all the stack components might be used on a specific stack setup. Some of the stack components are listed below:
cron.php
as soon as the file is available, and it will attempt to do so every 60 seconds since its last run. This container is used for any other CLI execution required during development/etc/hosts
the entry docker.local
to point to your machine's ip (it might be 127.0.0.1
if running the stack locally, or the LAN static ip address of the VM running Docker. If using the Debian VirtualBox image provided at the bottom of this README, the ip address is 10.10.10.10
)git clone https://github.com/esimonetti/SugarDockerized.git sugardocker
and enter sugardocker with cd sugardocker
Please leverage the utility script stack.sh that will help with automation of the most common stacks. The utility will also notify you if a new version of SugarDockerized is available.
For details about the hostnames and credentials of each of the infrastructure components, refer to Sugar Setup details.
installfromzip.sh
. Read more about installfromzip.sh
./data/app/
directory so that the repository is located in ./data/app/Mango/
build/build.sh
. Read more about build/build.sh
The main stacks work with Sugar version 9.0 and all its platform requirements. Additional stacks are aligned with the platform requirements of version 8.0, 7.9 and stacks for Sugar Cloud only versions for local development only.
To verify all components hostnames just run docker ps
when the stack is up and running.
Please note that on this setup, only the web server or the load balancer (if in single web server or cluster stack) and the database can be accessed externally. Everything else is only allowed within the stack components.
Apache web servers have enabled:
Apache web servers have PHP with enabled:
xdebug.sh
.
If you use an IDE such as PHPStorm, you can setup DBGp Proxy under the menus Preference -> Language & Framework -> PHP -> Debug -> DBGp Proxy. Example settings are available in the screenshot below:
Debug with Xdebug Helper
If you use Chrome as a browser, you can install the extension Xdebug helper. When ready to debug, click the debug button on the Xdebug helper, and click on "Start listening for PHP Debug Connections" within PHPStorm
Debug with Postman
It is possible to debug a specific API endpoint through Postman leveraging a similar approach. In this example, we are going to debug the login authentication api endpoint rest/v11_1/oauth2/token. The first step is to add the cookie "XDEBUG_SESSION" in Postman. The same cookie is set through the Xdebug helper, and the keyword is referenced on the PHPstorm settings and on Xdebug PHP server side settings as well. See screenshots below:
Session storage is completed leveraging the Redis container.
If you notice that your Elasticsearch container is not running (check with docker ps
), it might be required to tweak your Linux host settings.
To be able to run Elasticsearch version 5 and above, it is needed an increase of the maximum mapped memory a process can have. To complete this change permanently run:
echo "vm.max_map_count=262144" | sudo tee -a /etc/sysctl.conf
Alternatively the limit can be increased runtime with:
sudo sysctl -w vm.max_map_count=262144
images/php/XY/apache/
- Image for Apache with PHP version X.Ximages/php/XY/cron/
- Image for PHP version X.Y, for background jobs and any CLI needimages/mysql/XY/
- Image for MySQL version X.Yimages/elasticsearch/XY/
- Image for Elasticsearch X.Yimages/permissions/
- Image for permissions fixing containerimages/loadbalancer/
- Image for Apache load balancerimages/jmeter/
- Image for Jmeterimages/sidecar-build/
- Image for building Sidecar's javascriptimages/traefik/
- Traefik image to expose Sugar within the local network when using a VMimages/ldap/
- OpenLDAP imageMost images are currently leveraging Debian linux.
All persistent storage is located within the ./data
directory tree within your local checkout of this git repository.
./data/app/sugar/
. Within the web servers and the cronjob server the location is /var/www/html/sugar/
. Everything within ./data/app/
can be accessed through the browser, but the Sugar instance files have to be within ./data/app/sugar/
./data/mysql/XY/
./data/elasticsearch/XY/
./data/redis/
./data/ldap/
Do not change the permissions of the various data subdirectories, as it might cause the system to not work correctly.
This setup is designed to run only a single Sugar instance. It also requires the application files to be exactly on the right place for the following three reasons:
For the above reasons the single instance Sugar's files have to be located inside ./data/app/sugar/
(without subdirectories), for the stack setup to be working as expected.
If you do need multiple instances, as long as they are not running at the same time, you can leverage the provided tools to replicate and swap the data directories.
To help with development, there are a set of tools provided within the utilities
directory of this repository. Some of the scripts are mentioned below.
./utilities/xdebug.sh [status | start | stop]
./utilities/xdebug.sh status
xDebug status
PHP 7.1.33 (cli) (built: Nov 22 2019 18:28:25) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2018 Zend Technologies
with Zend OPcache v7.1.33, Copyright (c)1999-2018, by Zend Technologies
./utilities/xdebug.sh start
Start xDebug
6c9a9862b60c
PHP 7.1.33 (cli) (built: Nov 22 2019 18:28:25) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2018 Zend Technologies
with Zend OPcache v7.1.33, Copyright (c) 1999-2018, by Zend Technologies
with Xdebug v2.9.2, Copyright (c) 2002-2020, by Derick Rethans
./utilities/xdebug.sh stop
Stop xDebug
6c9a9862b60c
PHP 7.1.33 (cli) (built: Nov 22 2019 18:28:25) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2018 Zend Technologies
with Zend OPcache v7.1.33, Copyright (c) 1999-2018, by Zend Technologies
Due to performance impact, Xdebug is disabled by default. This script prompts you to activate Xdebug, check if Xdebug is activated, or disable it.
If you do not want to configure DBGp Proxy when running the script, you can specify the second argument change-ip
. In this case, the script will change the xdebug.remote_host
option to your local IP address.
./utilities/xdebug.sh start change-ip
Start xDebug
New IP of remote_host: 192.168.0.105
6c9a9862b60c
PHP 7.1.33 (cli) (built: Nov 22 2019 18:28:25) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2018 Zend Technologies
with Zend OPcache v7.1.33, Copyright (c) 1999-2018, by Zend Technologies
with Xdebug v2.9.2, Copyright (c) 2002-2020, by Derick Rethans
./utilities/setownership.sh
All directories and files within "data" are now owned by uid:gid 1000:1000
It sets the correct ownership of the data directories
./utilities/stack.sh 80 down
./utilities/stack.sh 80 down
stacks/sugar8/php71.yml down
Stopping sugar-cron ... done
Stopping sugar-web1 ... done
Stopping sugar-redis ... done
Stopping sugar-mysql ... done
Stopping sugar-elasticsearch ... done
Removing sugar-cron ... done
Removing sugar-web1 ... done
Removing sugar-redis ... done
Removing sugar-mysql ... done
Removing sugar-permissions ... done
Removing sugar-elasticsearch ... done
Removing network sugar8_default
No stopped containers
It helps to take the default stack for the sugar version passed as a parameter, up or down. It expects two parameters: version number (eg: 80, 90 etc) and up/down.
Have a look at the configuration file ./utilities/stacks.conf
, to know all the available stack combinations for the script. For some of the main stacks is available the "local" version of the stack, that allows local modification of settings and local docker image building.
./utilities/runcli.sh "php ./bin/sugarcrm password:weak"
It helps to execute a command within the CLI container. It requires the stack to be on
This script is a wrapper for the external Sugar CLI tool called Toothpaste. The tool is downloaded automatically during its first use from packagist through composer. To update it periodically, run ./utilities/toothpaste/install.sh
. To understand more about the featureset, either read more on the repository, or just run ./utilities/toothpaste.sh list
to list all the commands currently available.
This tool is also used to run a system repair throughout the system scripts.
./utilities/backup.sh 802_2018_11_21
Backing up sugar to "backups/backup_802_2018_11_21"
[sudo] password for docker:
Application files backed up on backups/backup_802_2018_11_21/sugar
Database backed up on backups/backup_802_2018_11_21/sugar.sql
It takes a snapshot of sugar files on backups/backup_802_2018_11_21/sugar
and a MySQL database dump on backups/backup_802_2018_11_21/sugar.sql
.
The script assumes that the database name is sugar and the web directory is sugar as well. The script does not take backups of Elasticsearch and/or Redis.
./utilities/restore.sh 802_2018_11_21
Restoring sugar from "backups/backup_802_2018_11_21"
sugar-permissions
Application files restored
Database "sugar" dropped
Database restored
Debug: Entering directory .
Repairing...
Repair completed in 9 seconds.
System repaired
It restores a previous snapshot of sugar files from backups/backup_802_2018_11_21/sugar
and of MySQL from backups/backup_802_2018_11_21/sugar.sql
The script assumes that the database name is sugar and the web directory is sugar as well. The script does not restore Elasticsearch and/or Redis.
This script installs the jmeter components present on the performance repository. Access to the repository is needed, if you are a Sugar Partner or Customer you can request access by mailing: [email protected]
This script runs the jmeter test scenario. To allow authentication, users with the format user:user1 pass:user1 should be created in advance.
$./utilities/jmeter/run.sh "ant customerTarget -file build.xml -DHOST_HEADER=10.10.10.10 -DSERVER=10.10.10.10 -DTIMER_DELAY=5000 -DTHREADS=5 -DITERATION_NUM=8 -DREST_ENDPOINT=/sugar/rest/v11_1 -DTHREAD_RAMP_UP=120"
The following arguments should be adapted to target server:
This script requires a Sugar zip package, and the zip package needs to be on the host running SugarDockerized.
If the installable zip package is located within the local directory ~/installable/SugarEnt-9.0.2.zip
the install command would be:
./utilities/installfromzip.sh ~/installable/SugarEnt-9.0.2.zip
The script will perform all the steps required to silently install the software within the SugarDockerized stack currently running. To know more about additional custom configuration parameters and actions that can be used/performed during the silent installation refer to script build/silentinstall.sh.
This script is most likely for Sugar Employees only.
The script requires a clone of Sugar's git repository within ./data/app/Mango
with the index.php
located within ./data/app/Mango/sugarcrm/index.php
.
If we want to build the application version 10.0.0 Enterprise, the build command would be:
./utilities/build/build.sh ent 10.0.0
The script will perform all the steps necessary to build the current branch of the software and also perform its silent installation. To know more about additional custom configuration parameters and actions that can be used/performed during the silent installation refer to script build/silentinstall.sh.
The script is leveraged by both installfromzip.sh
and build/build.sh
to install silently Sugar on the current stack.
This script allows a certain degree of flexibility by allowing the following customisations:
config_override.php
- to provide additional settings that might be needed for your Sugar installationThe custom installation configuration has to be located in ./data/app/configs/install_config_custom.php
. This script is optional. An empty and initialised version of this file will be generated if it does not exist.
An example of what an installation configuration looks like, is below:
<?php
$config_override = [
'setup_license_key' => '123123123123123123123123123123123123123',
'demoData' => 'yes',
];
As an example, if SugarDockerized is used only for the web server parts, and for the storages (database, search, cache) you decide to use native AWS components, you could easily override the installation settings required to do so. The full list of available setup options can be found on SugarDockerized ./utilities/configs/install_config.php
.
config_override.php
The custom config_override.php
has to be located in ./data/app/configs/config_override_custom.php
. This script is optional. An empty and initialised version of this file will be generated if it does not exist.
An example of what a custom config_override.php
looks like, is below:
<?php
$config_override = [];
$config_override['passwordsetting']['minpwdlength'] = 1;
$config_override['passwordsetting']['oneupper'] = 0;
$config_override['passwordsetting']['onelower'] = 0;
$config_override['passwordsetting']['onenumber'] = 0;
$config_override['passwordsetting']['onespecial'] = 0;
$config_override['passwordsetting']['SystemGeneratedPasswordON'] = 0;
$config_override['passwordsetting']['forgotpasswordON'] = 0;
$config_override['moduleInstaller']['packageScan'] = true;
The above example disables automated password generation, forgot password functionality and reduces the complexity of enforced password policy to just 1 character for testing purposes. It also enables package scanner on the instance, to make sure all packages installed on the application through its api/UI can also be installed on SugarCloud.
The custom initialisation PHP script that is executed after the silent installation has to be located in ./data/app/custominitsystem.php
. This script is optional.
An example of a useful initilisation script is below:
<?php
// settings
$set = [
'email-admin' => '[email protected]',
'email-user' => '[email protected]',
'tzone' => 'Australia/Sydney',
'date' => 'd/m/Y',
'time' => 'H:i',
'name-format' => 's f l',
];
echo 'Updating admin user' . PHP_EOL;
$admin = \BeanFactory::newBean('Users');
$admin->getSystemUser();
$admin->first_name = 'admin';
$admin->last_name = 'admin';
$admin->email1 = $set['email-admin'];
$admin->cookie_consent = 1;
$admin->save();
$admin = \BeanFactory::getBean('Users', $admin->id);
$admin->setPreference('viewed_tour', 1);
$admin->setPreference('timezone', $set['tzone']);
$admin->setPreference('datef', $set['date']);
$admin->setPreference('time', $set['time']);
$admin->setPreference('ut', 1);
$admin->setPreference('default_locale_name_format', $set['name-format']);
$admin->savePreferencesToDB();
echo 'Creating test user' . PHP_EOL;
$u = \BeanFactory::newBean('Users');
$u->user_name = 'test';
$u->first_name = 'test';
$u->last_name = 'test';
$u->user_hash = \User::getPasswordHash('test');
$u->status = 'Active';
$u->email1 = $set['email-user'];
$u->cookie_consent = 1;
$u->save();
$u = \BeanFactory::getBean('Users', $u->id);
$u->setPreference('viewed_tour', 1);
$u->setPreference('timezone', $set['tzone']);
$u->setPreference('datef', $set['date']);
$u->setPreference('time', $set['time']);
$u->setPreference('ut', 1);
$u->setPreference('default_locale_name_format', $set['name-format']);
$u->savePreferencesToDB();
echo 'Setting default mail server to sugar-smtp' . PHP_EOL;
$oe = \BeanFactory::newBean('OutboundEmail');
$oe->mail_smtpserver = 'sugar-smtp';
$oe->mail_smtpport = 25;
$oe->mail_smtpssl = 0;
$oe->saveSystem();
The above script configures specific settings of the Sugar System user and also creates a new test user. The script can be extended and improved to suit the specific needs.
To be able to achieve this consistently, it is recommended to leverage the single web server stack.
By running the command docker logs -f sugar-web1
it is then possible to tail the output from the access and error log of Apache and/or PHP. To view just the errors of Apache and/or PHP it is possible to run the command docker logs -f sugar-web1 1>/dev/null
. The same approach applies to the sugar-cron
container for cron and cli debugging. It applyes to any Docker container in general.
You would just need to run again the permissions docker container with docker start sugar-permissions
. The container will fix the permissions and ownership of files for you and then terminate its execution.
Apache and Cron run as the sugar
user. Se the following options on config_override.php
$sugar_config['default_permissions']['user'] = 'sugar';
$sugar_config['default_permissions']['group'] = 'sugar';
The application installs few scripts built to facilitate the system's repair from command line. The scripts will wipe the various caches (including OPcache and Redis if used). The cron container from which the repair runs, has also been optimised to speed up the repairing processing. To run the repair from the docker host, assuming that the repository has been checked out on sugardocker execute:
cd sugardocker
./utilities/repair.sh
The actual code for repair.sh
leverages the toothpaste.sh
script mentioned above.
Add on config_override.php
the following options:
$sugar_config['external_cache_disabled'] = false;
$sugar_config['external_cache_disabled_redis'] = false;
$sugar_config['external_cache']['redis']['host'] = 'sugar-redis';
Make sure there are no other caching mechanism enabled on your config/config_override.php combination, otherwise set them as disabled = true.
To run a PHP script execute something like the following sample commands:
docker@docker:~/sugardocker$ ./utilities/runcli.sh "php ../myAppTestCliScript.php"
docker@docker:~/sugardocker$ ./utilities/runcli.sh "cd ../toothpaste && ./vendor/bin/toothpaste local:system:repair --instance ../sugar"
docker@docker:~/sugardocker$ ./utilities/runcli.sh "whoami"
sugar
docker@docker:~/sugardocker$ ./utilities/runcli.sh "pwd"
/var/www/html/sugar
If needed, sudo is available as well without the need of entering a password. Just make sure the permissions and ownership (user sugar
) is respected.
Both XHProf extension and Tideways extensions are configured in most stacks.
To enable profiling:
config_override.php
specific settings (see below based on the stack extension)XHProf Sugar config_override.php
configuration:
$sugar_config['xhprof_config']['enable'] = true;
$sugar_config['xhprof_config']['log_to'] = '../profiling';
$sugar_config['xhprof_config']['sample_rate'] = 1;
$sugar_config['xhprof_config']['flags'] = 0;
Tideways Sugar config_override.php
configuration:
$sugar_config['xhprof_config']['enable'] = true;
$sugar_config['xhprof_config']['manager'] = 'SugarTidewaysProf';
$sugar_config['xhprof_config']['log_to'] = '../profiling';
$sugar_config['xhprof_config']['sample_rate'] = 1;
$sugar_config['xhprof_config']['flags'] = 0;
Make sure new files are created on ./data/app/profiling/
when navigating Sugar. If not, ensure that the directory permissions are set correctly so that the sugar
user can write on the directory.
Please note that profiling degrades user performance, as the system is constantly writing to disk profiling information and tracking application stats. Use profiling only on replica of the production environment.
./data/app/performance/
config_override.php
settings available on ./data/app/performance/
are kept as is (<?php $config['profile_files_dir'] = '../profiling';
)$sugar_config['external_cache_disabled'] = false;
$sugar_config['external_cache_disabled_redis'] = false;
$sugar_config['external_cache_force_backend'] = 'redis';
$sugar_config['external_cache']['redis']['host'] = 'sugar-redis';
$sugar_config['external_cache_disabled_wincache'] = true;
$sugar_config['external_cache_disabled_db'] = true;
$sugar_config['external_cache_disabled_smash'] = true;
$sugar_config['external_cache_disabled_apc'] = true;
$sugar_config['external_cache_disabled_zend'] = true;
$sugar_config['external_cache_disabled_memcache'] = true;
$sugar_config['external_cache_disabled_memcached'] = true;
$sugar_config['cache_expire_timeout'] = 600; // default: 5 minutes, increased to 10 minutes
$sugar_config['disable_vcr'] = true; // bwc module only
$sugar_config['disable_count_query'] = true; // bwc module only
$sugar_config['save_query'] = 'populate_only'; // bwc module only
$sugar_config['collapse_subpanels'] = true; // 7.6.0.0+
$sugar_config['hide_subpanels'] = true; // bwc module only
$sugar_config['hide_subpanels_on_login'] = true; // bwc module only
$sugar_config['logger']['level'] = 'fatal';
$sugar_config['logger']['file']['maxSize'] = '10MB';
$sugar_config['developerMode'] = false;
$sugar_config['dump_slow_queries'] = false;
$sugar_config['import_max_records_total_limit'] = '2000';
$sugar_config['verify_client_ip'] = false;
Tweak the above settings based on your specific needs.
You can run PHPMyAdmin in a container to get access to the database tables.
Pull the image
docker pull phpmyadmin/phpmyadmin
Find the network name
docker inspect sugar-mysql -f "{{json .NetworkSettings.Networks }}"
Note the network name from the result
{"sugar9_default":{"IPAMConfig":null,"Links":null,"Aliases":["25cea53d92b9","mysql"],"NetworkID":"a5a4d323a0a423ad81512c189f73a5b44195a72708e0d48819cb1bd3c89ff5ba","EndpointID":"ea35c217dc0a8b23c09dbb1a46ca29de710dde7fe954413e92967bfc50808d43","Gateway":"172.20.0.1","IPAddress":"172.20.0.4","IPPrefixLen":16,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"02:42:ac:14:00:04","DriverOpts":null}}
Run phpmyadmin and forward port 80 to 8080 on localhost. Substitute your network name.
docker run --network sugar9_default --name myadmin -d -e PMA_HOST=sugar-mysql -p 8080:80 phpmyadmin/phpmyadmin
Go to PhpMyAdmin
These stacks have been built on a Mac platform, that is known to not perform well with Docker mounted volumes.
Personally I run Docker on a Debian based minimal VirtualBox VM with fixed IP, running a NFS server. I either mount NFS on my Mac when needed or SSH directly into the VM. The Debian Docker VirtualBox VM for Mac is available here with its latest downloadable version here.
Alternatively, (...if you are brave enough to run Docker locally on a Mac) you can attempt to use mounted volumes for the data storage (Redis, MySQL and Elasticsearch) and the delegated option. An example of this setup working, can be found here stacks/sugar9/php73-mac.yml
and it can be initiated with ./utilities/stack.sh 90-mac up
.
To compare performance between Mac and Linux VM on Mac we can use the Toothpaste utility as follows: ./utilities/toothpaste.sh "local:analysis:fsbenchmark --instance ../sugar"
.
Mac with 90
stack:
Read speed: 640.50 KB/s
Write speed: 1,407.49 KB/s
Execution completed in 840.26 seconds.
Mac with optimised 90-mac
stack:
Read speed: 2,455.70 KB/s
Write speed: 1,735.60 KB/s
Execution completed in 333.20 seconds.
Linux VirtualBox VM on Mac with 90
stack:
Read speed: 114,875.19 KB/s
Write speed: 56,997.04 KB/s
Execution completed in 8.26 seconds.
The performance difference with the same exact test cannot be ignored, hence the recommendation to use a Linux VirtualBox VM. The above mentioned disk speeds make the Mac setup unusable.