Translation process automation with Symfony

Translations are a big part of website development, especially if one works on an international project. There are a lot of different solutions and packages for easier translation handling out there. If you ever used Symfony, you also probably used the Translation component. It is pretty simple to use and robust enough for most websites.

To use Symfony Translation Component, one must add a default language code (locale) and at least one fallback locale to project configuration. Default locale marks the default language to the application. Fallback locales is a list of all language codes that are used in the application. Fallback locales are also used if Symfony cannot find a translation for a message in the default locale. The Locale that is used at a given moment is usually stored in the request data.

Actual translations are arrays of messages stored inside files with special paths that Symfony Translator is programmed to find. One message, in general, consists of message ID and message translation. All message translations for one language are saved in a single file (can be .xml, .yml or .php file) with a language code before the extension of the file (eg.translations/messages.fr.yml)

The downside of the Translator component is the fact that you have to manually enter each locale you want to use and manually create all the translation files and translate the messages in there.

I worked on a project where we had multiple franchise accounts divided by county. When a new franchise is added, all the messages in the system need to be translated into the official language of that franchises’ country. In order to achieve this, you need to add the language locale to project configuration, create translation files with that language code and translate all the messages in that file, and that is after you get all the correct translation strings from the client. Since the client already had 10+ franchises and was planning on opening new ones, I didn’t really feel like doing this manually every time. Instead, we developed a feature that enables the client to add or remove languages on the platform, and translate all the messages in site administration.

This is the basic flow of the feature:

Adding a new language to the project:
1. Administrator adds a new language to the system through a form
2. New locale is added to project configuration automatically
3. Translation files are automatically generated
4. Administrator assigns a language to franchise (not covered in this tutorial since it’s project-related logic)
Translating messages through a form
1. Franchise owner or translator translate all the messages into their language
2. New messages are saved to translation files and can be used on frontend

The Backend implementation of every step in the flow is shown below. Please note that below is not the full code of the feature since it is too large and the structure is a bit complicated, so some classes here are either simplified or they implement some methods that are injected through dependencies in real life.

Adding a new language to the project

As mentioned above, to add a new language to a Symfony project, you need to add its locale to config.yml file. To automate this process, the first thing you need to do is create a Language entity to which you will map information about new language added:

class Language
{
   /**
    * @var int
    */
   private $id;

   /**
    * @var string
    */
   private $languageName;

   /**
    * @var string
    */
   private $languageCode;

   //Getters and setters
   ...
}

I validated the entity in the following way:

TranslationBundle\Entity\Language:
   constraints:
       - Symfony\Bridge\Doctrine\Validator\Constraints\UniqueEntity:
           fields: [languageCode]
   properties:
       languageCode:
           - NotBlank:  ~
           - Length:
               min: 2
               max: 2
           - Language: ~
       languageName:
           - NotBlank:  ~

I implemented this feature to be compatible with 2-letter ISO 639-1 language codes, but you can validate them to be compatible with any language code standard using Symfony’s Language constraint.

After the submit of language form, you need to add an event subscriber in which you will call methods for saving the locale to configuration file and generate translation files with that language code:

/**
* @param string $languageCode
*/
public function saveLanguageConfig(string $languageCode): void
{
   // Returns config file path
   $configPath = ProjectFolderFinder::findConfigFile();

   // Parses yaml file from path and dumps it into an array
   $config = FileGetter::parseYamlFile($configPath);

   // If the language code is not already under fallbacks part of config - add it
   if (!in_array($languageCode, $config['framework']['translator']['fallbacks'])) {
       array_push($config['framework']['translator']['fallbacks'], $languageCode);
   }

   // Write to file
   file_put_contents($configPath, Yaml::dump($config, 5, 4, 
       Yaml::DUMP_MULTI_LINE_LITERAL_BLOCK));
}

/**
* @param string $languageCode
*/
public function generateTranslationFiles(string $languageCode): void
{
   // First find all bundle paths that exist in the project
   $bundlePaths = ProjectFolderFinder::findProjectBundles();

   // Loop through all bundle paths and find ones that have
   // Resources/translation folder
   foreach ($bundlePaths as $bundle) {
       $translationsPath = 
           FileGetter::getFullFilePath('/'.$bundle."/Resources/translations");

       // Check if there are translation files in current bundle
       if (!file_exists($translationsPath)) { 
           continue; 
       }

       // Find all files that have .en.yml extension -> those
       // are english translations
       foreach (glob($translationsPath."/*.en.yml") as $englishTranslationFilePath)
       {
           // The path to file with english translations
           $filename = basename($englishTranslationFilePath, ".en.yml");

           // The name of the new file that needs to be created
           $newTranslationFileName =$filename.'.'.$languageCode.'.yml';

           // If the file already exists, don't create it
           if (file_exists($translationsPath.'/'.$newTranslationFileName)) {
                   continue;
           }

           // Get english translations
           $englishTranslations =
                   Yaml::dump(FileGetter::parseYamlFile($englishTranslationFilePath));

           // Generate new translations file and write english translations into it
           file_put_contents($translationsPath.'/'.$newTranslationFileName, 
                    $englishTranslations);
           
           // Add 0755 permissions to file
           chmod($translationsPath.'/'.$newTranslationFileName, 0755);
       }
   }
}

As user can also remove languages from the system, an analogous methods needs to be called from a subscriber that listens to DELETE events:

public function deleteLanguageConfig(array $languageCodes): void
{
   // Find config file
   $configPath = ProjectFolderFinder::findConfigFile();

   // Dump config file into an array
   $config = FileGetter::parseYamlFile($configPath);

   // Get existing fallbacks
   $fallbacks = $config['framework']['translator']['fallbacks'];

   // Remove wanted languages from fallback array
   foreach ($languageCodes as $languageCode) {
       if (($key = array_search($languageCode, $fallbacks)) !== false) {
           unset($fallbacks[$key]);
       }
   }

   // Write new fallback languages to config file
   $config['framework']['translator']['fallbacks'] = $fallbacks;
   file_put_contents($configPath, Yaml::dump($config, 5, 4, 
       Yaml::DUMP_MULTI_LINE_LITERAL_BLOCK));
}

/**
* @param string $fileExtension
*/
public function deleteTranslationFiles(string $fileExtension)
{
   // Find paths to all bundle folders
   $bundlePaths = ProjectFolderFinder::findProjectBundles();
   foreach ($bundlePaths as $bundle) {
       // Get translation folder path to current bundle
       $translationsPath = 
            FileGetter::getFullFilePath('/'.$bundle."/Resources/translations");
       if (!file_exists($translationsPath)) {
           continue;
       }

       // If the folder exists, delete all files with wanted extension in it
       $translationFilePaths = glob($translationsPath."/*".$fileExtension;
       foreach ($translationFilePaths as $translationFilePath) {
           unlink($translationFilePath);
       }
   }
}

At this point, a new language code has been added to the database and to configuration, and translation files have been generated, but all the translations inside are in English, which is the default language of the system.

Translating messages through a form

For this feature to make sense, a franchise owner or a translator user must be able to translate all the messages from site administration. To make this possible, a GET and PUT translations routes need to be developed.

All the translations are mapped to different models, according to their file name. Usually, you would put all the general info messages into messages.*.yml files, validation messages into validators.*.yml files, and email translations into email.*.yml files, where * is the language code. Since the translations are saved to different files, they are easier to handle if they are mapped to different models during (de)serialisation. Since I wanted this to be handled with item instead of collection operations, YamlTranslation is a model that contains a collection of messages models:

class YamlTranslation
{
    /**
     * @var ArrayCollection
     */
    private $messageTranslations;

    /**
     * @var ArrayCollection
     */
    private $validationTranslations;

    /**
     * @var ArrayCollection
     */
    private $emailTranslations;

    /**
     * @var string
     */
    private $languageCode;

    // Getters and setters
    ...
}

Where the messages models look like this:

class MessageTranslation
{
    /**
     * @var string
     */
    private $messageTranslation;

    /**
     * @var string
     */
    private $messageTranslationId;

   // Getters and setters
   ...
}

class ValidationTranslation
{
    /**
     * @var string
     */
    private $validationTranslation;

    /**
     * @var string
     */
    private $validationTranslationId;

   // Getters and setters
   ...
}

class EmailTranslation
{
    /**
     * @var string
     */
    private $emailTranslationId;
    
    /**
     * @var string
     */
    private $emailSubjectTranslation; 
    
    /**
     * @var string
     */
    private $emailMessageTranslation;

   // Getters and setters
   ...
}

TheGET action

To get all the translation linked to one language, all we need as a parameter for request is a language code. Since all the messages are in .yml format, they need to be transformed into an object and returned to frontend:

/**
* @param string $languageCode
* @return \Symfony\Component\HttpFoundation\JsonResponse
*/
public function getAction(string $languageCode)
{
   // Check if user has the permission to get this language translations
   $this->languageAccessChecker->validateCurrentUserCanEditLanguage($languageCode);

   // Find message,validation and email translations
   $messageTranslations = 
      $this->translationFileFinder->findMessageTranslationFiles($languageCode);
   $validationTranslations = 
      $this->translationFileFinder->findValidationTranslationFiles($languageCode);
   $emailTranslations = 
      $this->translationFileFinder->findEmailTranslationFiles($languageCode);

   // Map validation and email translations to response object
   $yamlTranslation = $this->yamlTranslationResponse->setYamlTranslation(
       $messageTranslations,
       $validationTranslations,
       $emailTranslations,
       $languageCode
   );

   // Returns JSON encoded 
   return $this->apiResponseService->getSuccessJSONResponse($yamlTranslation);
}

The TranslationFileFinder class is used to find all the translation files for one language. Note that you can have different file names for translation files (eg. validators, emails, messages…). You can find any translation file in the following way:

/**
* Class TranslationFileFinder
* @package TranslationBundle\Service
*/
class TranslationFileFinder
{
   /**
    * @param string $languageCode
    * @return array
    */
   public function findValidationTranslationFiles(string $languageCode): array
   {
       // Returns an array of translation file paths that have 'validators' in their 
       // name
       return $this->findTranslationFiles($languageCode, 'validators');
   }

   /**
    * @param string $languageCode
    * @return array
    */
   public function findMessageTranslationFiles(string $languageCode): array
   {
       // Returns an array of translation file paths that have 'messages' in their name
       return $this->findTranslationFiles($languageCode, 'messages');
   }

   /**
    * @param string $languageCode
    * @return array
    */
   public function findEmailTranslationFiles(string $languageCode): array
   {
       // Returns an array of translation file paths that have 'emails' in their name
       return $this->findTranslationFiles($languageCode, 'emails');
   }

   /**
    * @param string $languageCode
    * @param string $fileName
    * @return array
    */
   private function findTranslationFiles(string $langCode, string $fileName)
   {
       // Initialize an empty result array
       $results = [];

       // Find the paths to all bundle folders
       $bundlePaths = ProjectFolderFinder::findProjectBundles();

       foreach ($bundlePaths as $bundle) {
           
           // Get the path to translation folder in each bundle
           $translationsPath = 
            FileGetter::getFullFilePath ('/'.$bundle."/Resources/translations");

           // Check if there are translation files in current bundle
           if (!file_exists($translationsPath)) { 
               continue;
           }

           // Find the paths to all translation files with
           // wanted file name and language code inside the folder
           $translationFilePaths = 
               glob($translationsPath."/".$fileName.".".$langCode.".yml");
           foreach ($translationFilePaths as $translationFilePath) {

               // Dump yaml file into an array
               $translations = FileGetter::parseYamlFile($translationFilePath);

               // Merge translations from current file 
               // into existing results
               $results = array_merge($results, $translations);
           }
       }
       
       // Returns all translations with wanted file names and
       // language codes from all bundles
       return $results;
   }
}

After the messages are retrieved, a response object can be created:

/**
* Class YamlTranslationResponse
* @package TranslationBundle\Model\Response
*/
class YamlTranslationResponse
{
   /**
    * @var YamlTranslation
    */
   private $yamlTranslation;

   /**
    * @var TranslationFactory
    */
   private $translationFactory;

   /**
    * YamlTranslationResponse constructor.
    * @param TranslationFactory $translationFactory
    */
   public function __construct(TranslationFactory $translationFactory)
   {
       $this->yamlTranslation = $translationFactory->createYamlTranslation();
       $this->translationFactory = $translationFactory;
   }

   /**
    * @param array $messageTranslations
    * @param array $validationTranslations
    * @param array $emailTranslations
    * @param string $languageCode
    * @return YamlTranslation
    */
   public function setYamlTranslation(
       array $messageTranslations,
       array $validationTranslations,
       array $emailTranslations,
       string $languageCode
   ): YamlTranslation {
       // Set Language code
       $this->yamlTranslation->setLanguageCode($languageCode);

       // Loop through message translations and map them to MessageTranslation model
       foreach ($messageTranslations as $key => $messageTranslation) {
           $newMessageTranslation = 
               $this->translationFactory->createMessageTranslation();
           $newMessageTranslation->setMessageTranslation($validationTranslation);
           $newMessageTranslation->setMessageTranslationId($key);
           $this->yamlTranslation->addMessageTranslation($newMessageTranslation);
       }

       // Loop through validation translations and map them to
       // ValidationTranslation model
       foreach ($validationTranslations as $key => $validationTranslation) {
           $newValidationTranslation = 
               $this->translationFactory->createValidationTranslation();
           $newValidationTranslation->setValidationTranslation($validationTranslation);
           $newValidationTranslation->setValidationTranslationId($key);
           $this->yamlTranslation->addValidationTranslation($newValidationTranslation);
       }

       // Loop through email translations and map them to EmailTranslation model
       foreach ($emailTranslations as $key => $emailTranslation) {
           $newEmailTranslation = $this->translationFactory->createEmailTranslation();
           $newEmailTranslation->setEmailTranslationId($key);
           $newEmailTranslation
               ->setEmailSubjectTranslation($emailTranslation['email']['subject']);
           if (isset($emailTranslation['email']['message'])) {
               $newEmailTranslation
                   ->setEmailMessageTranslation($emailTranslation['email']['message']);
           }
           $this->yamlTranslation->addEmailTranslation($newEmailTranslation);
       }

      // Returns YamlTranslation model with all mapped values 
      return $this->yamlTranslation;
   }

}

All messages are looped through and their values are mapped to different entities. After everything is mapped, YamlTranslationResponse object is returned to controller, encoded to JSON format and returned to the frontend. This action is used to fill in the edit translation form, which displays all messages that can be edited. For example, let’s say that the portuguese translations files are filled with the following info:

// src/AppBundle/Resources/translations/messages.pt.yml
paymentRequest.title: 'Solicita o de pagamento'
paymentRequest.receiver: Receptor
paymentRequest.sender: Emissor

// src/AppBundle/Resources/translations/validators.pt.yml
notNullMessage: 'Campo de preenchimento obrigat rio'
exactMessage: 'Este valor deve ter exatamente {{ limit }} caracteres.'
minMessage: 'Este valor muito curto. Dever ter {{ limit }} caracteres ou mais.'

// src/AppBundle/Resources/translations/emails.pt.yml
welcome:
    email: { subject: 'Bem-vindo', message: 'Para ser editado no formul rio de cadastro' }
reset_password:
    email: { subject: 'Pedido para restar senha', message: "Caro(a) %firstName%,\n\nEste um email autom  tico da plataforma' }

In that case, the JSON response for GET /yaml-translations/pt route would look like this:

{
  "languageCode": "pt",
  "messageTranslations": [
    {
      "validationTranslation": "Campo de preenchimento obrigat rio",
      "validationTranslationId": "notNullMessage"
    },
    {
      "validationTranslation": "Este valor deve ter exatamente {{ limit }} caracteres.",
      "validationTranslationId": "exactMessage"
    },
    {
      "validationTranslation": "Este valor muito curto. Dever ter {{ limit }} caracteres ou mais.",
      "validationTranslationId": "minMessage"
    },

    ...
  ],
  "validationTranslations": [
    {
      "validationTranslation": "Solicita o de pagamento",
      "validationTranslationId": "paymentRequest.title"
    },
    {
      "validationTranslation": "Receptor",
      "validationTranslationId": "paymentRequest.receiver"
    },
    {
      "validationTranslation": "Emissor",
      "validationTranslationId": "paymentRequest.sender"
    },

    ...
  ],
  "emailTranslations": [
    {
      "emailSubjectTranslation": "Bem-vindo",
      "emailMessageTranslation": "Para ser editado no formul rio de cadastro",
      "emailTranslationId": "welcome"
    },
    {
      "emailSubjectTranslation": "Pedido para restar senha",
      "emailMessageTranslation": "Caro(a) %firstName%,\n\nEste um email autom  tico da plataforma",
      "emailTranslationId": "reset_password"
    },
    ...
  ]
}

The PUT action

After the user filled in the translation, the frontend sends all the data in JSON format, request being the same structure as the response above. Received data gets transformed into a PHP object (not YamlTranslation object!) and all the translations are mapped to corresponding files:

/**
* @param Request $request
* @param $languageCode
*/
public function putAction(Request $request, $languageCode)
{
   // Map JSON to object
   $translations = json_decode($request->getContent());

   // Save translations to files
   $this->translationFileSaver->saveYamlTranslations(
       $languageCode,
       $translations
   );

   // Return success response
   return $this->apiResponseService->getSuccessJSONResponse('Translation updated');
}

Translations are mapped and saved to files in the following way:

/**
* Class TranslationFileSaver
* @package TranslationBundle\Service
*/
class TranslationFileSaver
{
   /**
    * @param string $languageCode
    * @param array $translations
    */
   public function saveYamlTranslations(
       string $languageCode, 
       array $translations
   ){
       // Check if there are message translations
       if (isset($translations->messageTranslations)) {

           // Process message translations
           $this->saveMessageTranslations(
               $languageCode,
               $translations->messageTranslations
           );
    
           // Remove cache
           $this->removeCacheDir();
       }

       // Check if there are validation translations
       if (isset($translations->validationTranslations)) {

           // Process validation translations
           $this->saveValidationTranslations(
               $languageCode,
               $translations->validationTranslations
           );

           // Remove cache
           $this->removeCacheDir();
       }

       // Check if there are email translations
       if (isset($translations->emailTranslations)) {

           // Process email translations
           $this->saveEmailTranslations(
               $languageCode,
               $translations->emailTranslations
           );

           // Remove cache
           $this->removeCacheDir();
       }
   }

    /**
     * @param string $languageCode
     * @param array $translations
     */
    private function saveValidationTranslations(
        string $languageCode, 
        array $translations
    ){
        // Call saveTranslations method with 'validators' file path
        $this->saveTranslations($languageCode,$translations,'validators');
    }

    /**
     * @param string $languageCode
     * @param array $translations
     */
    private function saveMessagesTranslations(
        string $languageCode, 
        array $translations
    ){
        // Call saveTranslations method with 'messages' file path
        $this->saveTranslations($languageCode,$translations,'messages');
    }

    /**
     * @param string $languageCode
     * @param array $translations
     * @param string $fileName
     */
    private function saveTranslations(
        string $languageCode, 
        array $translations,
        string $fileName
    ){
       // Find bundle paths
       $bundlePaths = ProjectFolderFinder::findProjectBundles();

       foreach ($bundlePaths as $bundle) {

           // Find translation folder path in each bundle
           $translationsPath = 
               FileGetter::getFullFilePath('/'.$bundle."/Resources/translations");
           if (!file_exists($translationsPath)) {
               continue;
           }

           // Find all the translation files inside folder
           // with wanted file names
           $translationFilePaths = 
                   glob($translationsPath.$fileName.$languageCode.".yml");
           foreach (translationFilePaths as $translationFilePath) {
               // Update all the translations inside the file
               $this->saveTranslation($translationFilePath,$translations);
        }
    }

    private function saveTranslation(string $translationFilePath, array $translations)
    {
        // Get file content
        $fileContent = FileGetter::parseYamlFile($translationFilePath);
        
        // Loop through every translation from request
        foreach ($translations as $translation) {

            // If the translation ID exists in the current 
            // file - replace it with the one from request
            if (key_exists($translation->translationId, $fileContent)) {
                $fileContent[$translation->validationTranslationId] = 
                    $translation->translation;
            }
        }
        
        // transform translations object to yaml
        $translationsToSave = Yaml::dump($fileContent);

        // write translations to file
        file_put_contents($translationFilePath, $translationsToSave);
    }

   /**
    * @param string $languageCode
    * @param array $translations
    */
   private function saveEmailTranslations(
       string $languageCode, 
       array $translations
   ){
       $bundlePaths = ProjectFolderFinder::findProjectBundles();

       foreach ($bundlePaths as $bundle) {
           $translationsPath = 
               FileGetter::getFullFilePath('/'.$bundle."/Resources/translations");
           if (!file_exists($translationsPath)) { 
               continue; 
           }

           // Find all the translation files inside folder
           // with wanted file names
           $translationFilePaths = 
                   glob($translationsPath."/emails.".$languageCode.".yml");
           foreach $translationFilePaths as $translationFilePath) {
               $fileContent = FileGetter::parseYamlFile($translationFilePath);

               // Loop through translations from request
               foreach ($translations as $translation) {
                   if (!key_exists($translation->emailTranslationId, $fileContent)){
                       continue;
                   }
                   
                   // Find path to email array key in the file
                   $emailKeyPath =
                      $fileContent[$translation->emailTranslationId]['email'];

                   // Replace the subject of the email in the file
                   // with the one from the request
                   $emailKeyPath['subject'] = $translation->emailSubjectTranslation;
                           if (isset($emailKeyPath['message'])) {

                               // Replace the body of the email in the file
                               // with the one from the request
                               $emailKeyPath['message'] =
                                   $translation->emailMessageTranslation;
                           }
                   }

                   // Dump translation object into yaml
                   $translationsToSave = Yaml::dump($fileContent);

                   // Write to file
                   file_put_contents($translationFilePath, $translationsToSave);
               }
           }
       }
   }

   private function removeCacheDir()
   {
       // Find cache folder
       $cacheDir = dirname(__FILE__, 4).'/var/cache';
       if (file_exists($cacheDir)) {

           // Add 0777 rights to cache folder
           chmod($cacheDir, 0777);
           $fs = new Filesystem();

           // Delete cache folder
           $fs->remove($cacheDir);
       }
   }
}

Note that cache needs to be removed for changes to take place, since all the messages served to the frontend are mapped in cache files.

This implementation is done on Symfony 3.3 and .yml translation files, but should be easy enough to translate to newer versions of Symfony or .xml translation files. It has been almost 2 years since this feature is up and running on the production environment, and so far there were no larger issues with it. If a new translation message is added, you just paste it at the end of each translation file and notify the client that there are new messages to translate, which makes development time much shorter.

Also, if the client is not satisfied with some messages on their site, they can easily change it themselves without having to ask you.