Introduction

X2Engine provides the means to send email, natively, within the web application. Furthermore, with Professional Edition, the local MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4. on the web server (if any) can be configured to allow X2Engine to receive emails and automatically associate them with the addressee's contact record.

Configuring SMTP Accounts

This section describes the current, recommended method of sending email in X2Engine. It makes use of a credentials storage system X2Engine features, which encrypts passwords and other authentication data at the database storage level. It is currently only used for storing email account passwords, but may in the foreseeable future be extended to store other types of passwords. To access the credentials manager, click "Manage Apps" under the user menuThe menu accessed by clicking on one's own username in the far upper-right-hand corner of the application..

Creating Credentials Records

The create page for a generic email account

To create a new email account, first select a type in the dropdown menu next to the Add New button. Then, click Add New. The easiest type of email account to add is Google Mail (i.e. GMail or email accounts offered via Google Apps for businesses). For such email accounts, SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption. server data such as the host name, port, and security type are already set, and all you have to enter is your username, password, and sender name. For all other types of email, first contact your email hosting provider or peruse their documentation on configuring email client programs (i.e. Outlook / Thunderbird) for use with their email service. The essential information that you need:

• Server: host name, i.e. smtp.email.com
• Port: a 2-3 digit number, i.e. 25 for unsecured SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption., or 465 (legacy) or 587 for secure SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption..
• Security Type: TLS[[wikipedia:Transport Layer Security]] a method of connecting to a remote network host that encrypts traffic on any given network protocol that uses it./SSL(a.k.a Transport Layer Security) a method of connecting to a remote network host that encrypts traffic on any given network protocol that uses it.. In most cases these are interchangeable.
• User name/Password: If the SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption. server does not use authentication, leave these blank. Otherwise, set them according to the username/password used for logging in to the email

Creating System-Owned Credentials

One may wish to have all users use a specific email account/address for email blasting, or set a sender account for system notifications, or responses to service case submitters (etc). One can do so by creating publicly-usable email accounts. Such accounts will be usable by all users, but may only be viewed or edited by administrators, and thus are ideal for such purposes as they aren't necessarily attached to / identified with any given user.

To designate an account as system-owned, set the Owner to be "System" when creating it.

Creating Non-Private Credentials

Non-administrators also have the ability to create credentials that can be used by administrators, in the rare case where it may be desirable for the owner to send mail on their behalf. To do so (allowing administrators to masquerade as the other user when sending email), that user must consent to allow their email account to be used by other people, by un-checking the Private checkbox. Note, however, that this will not allow administrators to view or edit the credentials, only to use them as a delivery method option when sending email.

Setting Personal Default Credentials

Setting one's own personal default credentials

It is usually desirable to have a preferred email account for use in X2Engine, while at the same time having the option to send email using the system-wide (legacy) method, or via some other email account set up through the credentials manager.

Next to each credentials record, there will be a "pill" with a checkbox in it that says "Set as my default", or "Set as default" (when system-owned and being viewed by an administrative user). When the box is checked, an "apply" button will appear to enact the change. After clicking "apply" a green pill label appears on the record to denote that it has been selected as default. What this means is that whenever one opens the inline email widget (i.e. on a contact view, by clicking the email button), the "send as" field will be set to the selected account by default. This way it still allow switching email accounts on-the-fly, but removes an extra step that would otherwise be necessary to remember whenever sending an email.

Setting System Default Credentials

Setting default credentials for the system and/or oneself

All users have the ability to set system-owned credentials as their personal defaults. However, administrative users can also designate the credentials for system use purposes, i.e. bulk email and system notifications. For each credentials record owned by the system, when being viewed as the administrator, the pill will say "set as default" instead. When the box is checked, it will display a dropdown menu of system roles for which the account isn't already selected to use by default. After selecting the appropriate role to apply it as default for, and clicking "apply" as one would for setting personal defaults, it will be set as the default email account to use for that purpose.

Note, also, setting defaults for system roles in the credentials manager is just another way of changing settings in "Email Server Configuration" (accessed from within the Admin panel), and vice versa. These settings are actually synchronized. Some definitions of system uses:

Bulk Email Account

Will be selected by default when users create a new campaign (overrides the users' personal default)

Service Case Email Account

Used when sending responses to service case submitters.

System Response Emailer

Used when sending responses to web leads, and is selected by default in X2Flow's send-an-email actions (but can be changed).

System Notification Emailer

Used for miscellaneous notification tasks, most notably, sending "broadcast event" emails to targeted users when users broadcast an event, select users to broadcast it to, and enable the option to notify users by email.

Using The Legacy (system-wide) Email Delivery Methods

Apart from using separate, third-party SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption. accounts for each user (and system purpose), there is a way to use a singular email delivery method for all emails sent via X2Engine. However, this method is not recommended, unless your web server also serves as your mail server. It is very much considered a legacy feature.

There are four options for methods of sending email, which can be found in the Outbound Email Server section in Email Settings under the Email Configuration section in the Admin page. These methods are:

1. PHP Mail
2. Sendmail
3. QMail
4. SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption.

If any of these aren't available, they won't be listed in the Method menu.

PHP Mail, Sendmail and QMail each, while independent of any offsite, third-party service, require that an MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4. be available on the web server. The main difference is that PHP Mail uses whichever MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4. that PHP itself is configured to use, whereas the Sendmail and QMail methods are specific to the Exim4/Postfix and QMail MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4.'s (respectively).

For all instances of X2Engine Professional Edition cloud/ondemand, the methods available for sending mail include all of these except QMail.

Circumventing Spam Filters

While the non-SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption. methods for sending email within X2Engine may work immediately, a common problem encountered with them is that email goes into spam folders on the receiving end, or is not being received at all. This is especially likely to happen when the email addresses specified by users in their profiles (which get used as the sender address in the headers of mail sent by X2Engine) do not belong to the same domain name as the server hosting X2Engine. So, for example, it would be problematic if a user uses an email address in their X2Engine profile that ends in gmail.com, and the X2Engine system is hosted on example123.com

This is not a fault in X2Engine (or PHPMailer, which X2Engine uses for mail delivery). Rather, it is endemic to all web applications (i.e. Wordpress, Drupal, etc.) due to how sender address spoofing (telling the recipient that the email comes from somewhere else) is a common behavior among spammers. Furthermore, the general proliferation of email spam through various means (but especially PHP scripts running on web servers) has resulted in greatly elevated standards in the spam-detection mechanisms and heuristics of most email servers. This generally poses an entry barrier to getting around spam filters. It is thus recommended that, before sending emails using these legacy methods, measures are taken to ensure that mail sent will have the best chance of avoiding spam filters.

Work-around 1: Aliasing

This works by first creating mail forwarding aliases on the server, i.e. through a Virtualmin or CPanel control interface, or manually, by editing the MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4.'s virtual user aliases table (see their respective documentation for more information). Next, instruct all users to use these email addresses as their contact address in their X2Engine profile, so that when contacts reply, the response emails will be forwarded to the desired addresses (or even a list of addresses) specified by each alias.

Work-around 2: Migration

This, of course, is the most straightforward solution: to use the hosting provider's email service. However, it requires switching from one's existing email service/addresses to service and addresses provided by the hosting provider, and it is also extra effort.

Work-around 3: Third-party Service

Note, using this method is not recommended; see Configuring SMTP Accounts for the preferred method of using SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption. for email delivery.

Using this method, the entire organization that uses X2Engine sends email using a single email account managed by a third-party service that supports SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption.. Thus, sending the email and digital signing are handled by the third-party service, and getting caught in spam filters generally tends to be a smaller problem.

In most cases, this method will work. How it works is that the reply-to address in the email will be set to the email address of user who is sending the mail through X2Engine, while the actual mail system user is used to authenticate with the mail server. In such cases, the address of the mail user might still show up in one of the headers of the resulting email. However, this is known to not work with many email providers, most notably GMail, which (for security and anti-spam purposes) does not permit sending emails if the sender address differs from the address associated with the Google account that was used to authenticate. If this is the case, it is recommended to use separate SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption. accounts via the credentials manager.

Using DKIM

(see main article on Wikipedia)

This method, which requires server-end configuration, is a very secure and reliable method of distinguishing email as being of legitimate and trustworthy origin. Almost every reputable email service is configured to use a mail signing agent to add a special signature sent from such addresses, in addition to checking signatures of incoming emails. Emails that are digitally signed are far more likely to be recognized by mail servers worldwide as being from a reputable source, and thus less likely to go be caught in spam filters.

However, it still requires that the sender's email address be of the same domain as the email server. That is because the sender address cannot be "spoofed", because DKIM[http://www.dkim.org/ DomainKeys Identified Mail]: a system for digitally signing email that provides the means to securely verify that the email's origin is legitimate. was designed to prevent this sort of activity (which is common among spammers). Furthermore, if outgoing email is signed for domains not corresponding to the point of origin, it may actually make matters worse; many reputable mail services (including GMail) themselves use DKIM[http://www.dkim.org/ DomainKeys Identified Mail]: a system for digitally signing email that provides the means to securely verify that the email's origin is legitimate. to avoid their domain name being abused by address spoofing. Thus, if a DKIM[http://www.dkim.org/ DomainKeys Identified Mail]: a system for digitally signing email that provides the means to securely verify that the email's origin is legitimate. signature header is added to the email, and the signature does not match the DKIM[http://www.dkim.org/ DomainKeys Identified Mail]: a system for digitally signing email that provides the means to securely verify that the email's origin is legitimate. record of the sender's domain, it will be considered a forgery attempt and blocked/filtered by most email servers.^[1]

Troubleshooting SMTP

If, when attempting to send an email, you see an error message "SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption. Error: Could not connect to SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption. host" (from an exception thrown in PHPMailer), this indicates exactly what it says; it was unable to connect. There are a wide variety of reasons why an SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption. connection should fail, but they almost always fall into one of the following two categories:

1. Configuration of the connection and authentication
2. Local network environment

It is recommended to check them in that order. First, to test configuration, check:

1. The host name is correct
2. The SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption. port number is correct
3. The option to use TLS[[wikipedia:Transport Layer Security]] a method of connecting to a remote network host that encrypts traffic on any given network protocol that uses it. is enabled, if required by the email service provider
4. The username and password are correct and exactly as specified by the service provider (i.e. if the SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption. username is whole email address, it should be entered as the whole email address).

Testing the network configuration is more involved and will require logging into the server that hosts X2Engine via SSH or otherwise. One should first acquire the IP address of the SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption. host, via the command ping [smtp host] in a DOS or Unix shell, from any machine that is known to be able to access the mail server. You must then verify, locally on the X2Engine hosting server, that:

1. The SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption. host can be reached, i.e. there exists no valid network route and/or NAT rule allowing access to it, from the server on which X2Engine is hosted.
2. The SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption. hostname can be resolved (DNS[[wikipedia:Domain Name System]]: a naming system for computers, services, or any resource connected to the Internet or a private network (i.e. that identifies "google.com" with its internet protocol address) is available). Try ping again, on the server hosting X2Engine itself.
3. The port number is not being blocked by the ISP for outbound connections on the port.

If the first two requirements are satisfied and SMTP[[wikipedia:Simple Mail Transfer Protocol]]: a network protocol for sending email to a remote server, often involving password authentication and end-to-end SSL encryption. connections still fail, this is most likely the cause of the problem. One can also test this via port scanning or with telnet.

Configuring the "Email Dropbox" For Receiving Emails

This feature, available in X2Engine Professional Edition, provides the means to automatically create contacts and email-type action records by capturing emails from external email software, i.e. Outlook and GMail.

Note, if you are a customer of X2Engine Professional Cloud/On Demand: you do not need to set up the necessary prerequisites for this feature; that has been done for you. The feature should already be ready to use, and the alias will be dropbox@[yoursubdomain].x2vps.com. You will only need to configure the Email Dropbox if you are using Download/On Premise.

Setting it up requires a Linux/Unix server environment, a MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4., and knowledge of how to set up email aliases. There are two methods of integration: direct, and via API.

Using the Email Dropbox

As of the current version, there are three ways of using the email dropbox:

CC

When sending an email to a contact, include the email alias in the CC field, and the email will be attached to the contact as an outgoing email.

Forward

After receiving an email from a contact, forward it to the email alias. It will be attached to the contact as an incoming email.

Direct

Send an email directly to the alias, without "Fwd:" in the subject. A post in the social feed will be created using the contents of the email's body.

Case (coming in 4.0b)

Same as forward or CC, but include a user-defined code in the email body to specify that the email should be converted to a case in the Services module.

Direct Integration

This set-up method works by locally sending the email contents directly to X2Engine through the system console. It is more straightforward to configure, but requires that the MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4. and web server hosting X2Engine be on the same machine, virtual or physical.

1. Using CPanel, Webmin/Virtualmin, or any other web host administrative tool, create a mail alias on the server that begins with "dropbox@", i.e. "dropbox@yourdomain.com". (Note: you can also do this manually by editing the mail server's configuration files, although doing that is beyond the scope of this guide.)
2. Find the absolute path on the web server's filesystem to X2Engine, if you're not sure where it is already:
   1. Paste the following into a PHP file called "path.php", in the same directory as X2Engine:

      <?php echo realpath(dirname(__FILE__)); ?>
      

   2. Navigate to the file using a web browser
   3. Record the path, append "/email.php", and use the resulting string as the path to the capture script in the mail alias.
   4. Delete the file "path.php" from the server.
3. Create an email alias. Set its destination a pipe to a program: /usr/bin/php /path/to/X2Engine/email.php (versions up to 2.5.2) or /path/to/X2Engine/protected/yiic emaildropbox (versions 2.7 and later)
4. (X2Engine versions 2.7 and later) Change the permissions on the protected/runtime folder to 777 (all users can read/write).^[2]
5. Test the alias by sending a message to a fictitious name/email address (i.e. "Sue Doenimm" <test@example.com>) and CC-ing the mail alias.
6. Check in the Contacts module after sending the email, and if the fictitious contact was created, check to see that the email was attached properly to the contact record. If the contact was not created or the email was not imported, check the logs of the MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4. for any errors.
7. Test to see if your email software's forwarded message format is supported by forwarding an email from a contact to the mail alias.

API Integration

This method works by sending the email contents to X2Engine through the remote API, and thus permits the MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4. and X2Engine to live on different servers. Similar to the direct method, an alias will need to be constructed to receive the email and send it to a program, but in this case the program acts as a proxy by sending the contents to X2Engine via a web request. This method is more flexible, and it is far less likely to result in file permission issues (i.e. when the MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4. attempts to run the Yii console application), but takes slightly more effort to set up.

For information on API authentication, see Remote API: Authenticating.

The setup process is as follows:

1. Obtain the API script. Using your file/FTP manager of choice, navigate to the protected folder inside X2Engine, find the integration folder, and inside of that, navigate to the "MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4." folder.
2. Download/copy emailImport.php
3. Open the file in your editor of choice, and look for the following lines of code:

   ///////////////////////////
   // Configuration details //
   ///////////////////////////
   // Set this to the IP address or domain name of the server
   $host = '';
   // Set this to the protocol (use "https://" for an SSL-enabled web server)
   $proto = 'http://';
   // Set this to the URI on the web server of X2Engine, without the trailing slash.
   // So, if the login URL is "http://example.com/X2Engine/index.php/site/login",
   // this variable should be "/X2Engine"
   $baseUri = '';
   // Leave this null if the host specified by $host will resolve correctly.
   // Otherwise, if in an environment where (for instance) the domain does not resolve
   // properly, and the IP address must be used, but the CRM is on a specifically-named
   // virtual host on a shared IP, set this to the domain name of that host, and set
   // $host to the IP address of the web server.
   $hostName = '';
   $data = array(
   	'user' => '',
   	'userKey' => '',
   );
   

4. Change the variables according to their comments by putting the appropriate values in the (currently empty) pairs of single quotes. For instance, if your CRM is hosted at "https://crm.domain.com/crm", you will put "https://" in for $proto, "crm.domain.com" for $host, and "/crm" for $baseUri.
5. In the $data array, set the user and userKey to the API user and API key to be used (see: Remote API: Authenticating). So, for "admin" and key "aabbccddee":

   $data = array(
   	'user' => 'admin',
   	'userKey' => 'aabbccddee',
   );
   

6. Upload the script to the server to be handling email.
7. Ensure the script has proper ownership and permissions to be run by the MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4. user. If you're not sure what user the MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4. will run under when executing mail handling scripts, set the permissions on emailImport.php to 755 (all can read, owner/group can write, all can execute)
8. Create an email alias on the server to be handling email, which pipes to the program: /usr/bin/php /path/to/emailImport.php (replacing "/path/to" with the actual filesystem path where you uploaded the script).

Troubleshooting

Due to how this feature requires interaction between multiple software systems, a non-functioning email dropbox can have a very wide variety of possible causes.

Email is not captured

If no error email response is received from the mail handling server or email service provider, but the dropbox "silently" fails (by not capturing the email), verify the following:

• Your user profile email address matches the address from which you are sending email.
• If using the API method of integration, check:
  • The domain name of the X2Engine-hosting server is resolvable on the mail host
  • The CRM web host can be accessed from the mail host via a HTTP (web) request
  • If the CRM is on a server that is behind a firewall, port 80 (or 443, if using HTTPS) on the external IP address is forwarded to the CRM's web host
  • The API user and user key are correct in the API configuration
• If you have "Create Contacts from Emails" disabled, the recipient address of the email (or original sender, if you are forwarding the email) matches that of an existing contact in X2Engine

Email rejected with error message

Verify:

• Correct email server settings, DNS[[wikipedia:Domain Name System]]: a naming system for computers, services, or any resource connected to the Internet or a private network (i.e. that identifies "google.com" with its internet protocol address), and email alias
• Correct permissions/ownership on all scripts run by the MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4.

(API Method) No request to X2Engine showed up in the web server logs

This is due to one of the following problems:

1. The mail handling server never received the message
2. Any of the same conditions that would cause the MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4. to bounce the email back to the sender with an error message
3. The mail handling server couldn't connect to X2Engine's server

Ultimately, it boils down to there being no network connection between the mail handling server and X2Engine to begin with.

To investigate if #1 and #2, first look at the MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4.'s logs. They are typically stored in /var/log, and named maillog (RHEL/CentOS) or mail.log and mail.err (Debian/Ubuntu). If there is no record of mail received:

• Does the mail handling server have a fully-qualified domain name?
• Check the MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4.'s configuration. Is it configured to allow receiving mail for the system's FQDN[[wikipedia:Fully qualified domain name]]: a domain name that specifies its exact location in the tree hierarchy of the Domain Name System (DNS).?
• Is there a route to the mail host from the internet? And, are the necessary ports open for receiving mail?

Otherwise, if there are lines in the mail log(s) matching the alias, they will most often immediately indicate what went wrong. In such cases, the issue will typically be one of an improperly typed pipe alias (i.e. feeding directly to the script, rather than the PHP interpreter which runs the script), or permissions (meaning, Postfix child processes cannot read the import script file). Note that each node (parent directory) in the path to the script must be executable by the Postfix subprocess; else it cannot access anything within.

If, on the other hand (#3) there were no errors reported in the mail log for delivery to the alias, the problem is at the level of connecting between servers. Thus, you must verify:

• Is the domain name of the web server correct?
• Can the web server of the CRM be resolved to the proper IP address on the mail server?
• Can HTTP connections can be made to the server (i.e. verify the port isn't blocked, a valid route exists to the host, etc.)?
• Is the php_curl extension available in the command line environment on the mail handling server?

Response email regarding an unsupported forwarded message format

Forward the the email to X2Engine Customer Support, using the same email software that was used to test the feature. Due to the great diversity of email software and the inconsistency of forwarded/attached message formatting across platforms, the forwarded message capture may not immediately work with your email software of choice. However, per request, support for new forwarded message formats will be added in the next release.

Notes

1. ↑ DKIM[http://www.dkim.org/ DomainKeys Identified Mail]: a system for digitally signing email that provides the means to securely verify that the email's origin is legitimate. uses the RSA public/private key cryptographic technique to ensure that signatures are secure and cannot be forged. Thus, there is no legitimate way of circumventing this issue.
2. ↑ If you cannot do this for security purposes, but are a system administrator, you can instead change them to 770. If you do so, however, you must change the group ownership of the directory to the group under which the MTA[[wikipedia:Message transfer agent]]: an email relay service, i.e. Postfix, QMail or Exim4. spawns processes (typically "nobody" or "nogroup").