Saturday, 26 January 2019

Continuous Integration Using Jenkins with SalesforceDX | JWT-Based Flow


In this post we will talk about what is Continuous Integration and how to setup Continuous Integration Using SalesforceDX with Jenkins (Salesforce dx jenkins integration). We will use the OAuth JSON Web Token (JWT) bearer flow that’s supported in the Salesforce CLI. The JWT bearer flow supports the RSA SHA256 algorithm, which uses an uploaded certificate as the signing secret.

What Is Continuous Integration (CI) ?

Continuous integration (CI) is a DevOps software development best practice. Where developers regularly merge their code changes into a central repository, after that automated builds and tests are run. This best practice helps development teams to detect problems, identify bugs, and fix those problems and bugs before releasing changes to their customers. Please check our YouTube Recording for step by step process.

How to setup Continuous Integration Using Jenkins.

Step 1) Install Salesforce CLI. Here is link to download.

 

Step 2) Install Jenkins. Download from here.

  • Install Git Plugins.

 

Step 3) Create a Self-Signed SSL Certificate and Private Key

  • Set OPENSSL_CONF path  
    set OPENSSL_CONF=C:\openssl\share\openssl.cnf 
  • Generate an RSA private key :  
    openssl genrsa -des3 -passout pass:x -out server.pass.key 2048
  • Create a key file from the server.pass.key file
    openssl rsa -passin pass:x -in server.pass.key -out server.key
  • Request and generate the certificate :
    openssl req -new -key server.key -out server.csr
  • Generate the SSL certificate
    openssl x509 -req -sha256 -days 365 -in server.csr -signkey server.key -out server.crt

 

Step 4) Create Connected App for JWT-Based Flow

  • Created Connected App
    • Callback URL
      http://localhost:1717/OauthRedirect
    • Use digital signatures To upload your server.crt file.
  • Edit policy and select "Admin approved users are pre-authorized" to avoid "Not approved for access in salesforce" issue. 
  • Assign Connected App to user or System Admin profile.
  • Validate Authorize an Org Using the JWT-Based Flow  
    sfdx force:auth:jwt:grant --clientid {ADD_YOUR_CLIENT_ID} --jwtkeyfile server.key --username amit.salesforce21@gmail.com --instanceurl https://login.salesforce.com --setdefaultdevhubusername
    • --clientid  :- provide Consumer Key
    • --jwtkeyfile :- Absolute path to the location where you generated your OpenSSL server.key file
    • --instanceurl :-provide instanceurl if you are using sandbox.
    • --setdefaultdevhubusername :- Set Default dev hub User Name.
Sample

sfdx force:auth:jwt:grant --clientid KeyHere --jwtkeyfile c:\openssl\bin\server.key --username amit.salesforce21@gmail.com --instanceurl https://login.salesforce.com --setdefaultdevhubusername

Step 5) Configure the Jenkins environment Variable

  • Configure the the Server.key in the credential plugins. 
  • Set Environment variable:-
    • HUB_ORG_DH:- The username for the Dev Hub org, such as amit@gmail.com
    • SFDC_HOST_DH:- The login URL of the Salesforce instance that is hosting the Dev Hub org. The default is https://login.salesforce.com
    • CONNECTED_APP_CONSUMER_KEY_DH :- The consumer key that was returned after you created a connected app in your Dev Hub org.
    • JWT_CRED_ID_DH:- The credentials ID for the private key file that you stored in the Jenkins Admin Credentials interface
  • Install the Custom Tools Plugin into your Jenkins console, and create a custom tool that references the Salesforce CLI
The names for these environment variables are just suggestions. You can use any name as long as you specify it in the Jenkinsfile.

 

Step 6) Configure the Jenkins 

  • Add New Item.
  • Add Source and provide repository URL.  You can use URL as sample.
    https://github.com/amit-salesforce/SFDXProject
  •  Start a build.


We will talk about different option in Jenkins in our future post.

Please check our YouTube Recording for step by step process.


URL :- https://youtu.be/IvTlx6lBkQY


Check below post for more detail:-
  1. Salesforce DX Developer Guide
  2. Continuous Integration Using Salesforce DX

Check this post to learn more about Salesforce Dx. Here is recording for SalesforceDx Session in ApexHours. If you want to learn about Lightning Web Components. Check this post.


Please share your feedback and question.


Thanks,
Amit Chaudhary

47 comments:

  1. Hi Amit,
    Thanks for the detailed documentation.
    I am setting up the DX by following the instructions as mentioned but in the set up 4
    when i gave command as below

    sfdx force:auth:jwt:grant --clientid 3MVG9ZL0ppGP5UrDaftJUyCSnIoXbPpQSP2TVYPQixfytB2jssu.YO9dqgcJP_.q_.qhVa3Is6eUX_QvkjiXu --jwtkeyfile C:\Program Files\OpenSSL-Win64\bin\server.key --username ac@demo.com --instanceurl https://login.salesforce.com --setdefaultdevhubusername

    Giving the error as ! Unexpected argument Files\OpenSSL-Win64\bin\server.key

    Do you have any idea why is this error is happening?.

    Thanks,
    Rama Injety

    ReplyDelete
  2. Hi Amit,

    After i watched the Youtube Recording video i have figured out the issue.

    Once again thanks for sharing the information.

    It really helped me a lot.

    Thanks,
    Rama

    ReplyDelete
  3. Hi Amit,
    Thanks for the great post.
    I was trying to deploy to the target org from Github.
    the problem i am having with deployment although the build is successful but nothing added in the target org.
    Can you please help me where i am missing here.Seems like its not finding any data in manifest directory.
    Please help.
    My Github repository is https://github.com/bganguly1984/Salesforce.

    Thanks,
    Bhaskar

    ReplyDelete
    Replies
    1. Update your deployment command

      // need to pull out assigned username
      if (isUnix()) {
      rmsg = sh returnStdout: true, script: "${toolbelt} force:source:deploy --manifest manifest/package.xml -u ${HUB_ORG}"
      }else{
      rmsg = bat returnStdout: true, script: "\"${toolbelt}\" force:source:deploy --manifest manifest/package.xml -u ${HUB_ORG}"
      }

      Delete
  4. Replies
    1. Try to update your Jenkine like below file
      https://github.com/amit-salesforce/ApexHours/blob/master/Jenkinsfile

      Delete
  5. Hi Amit

    I am getting below error while building the job,

    Started by user admin
    Checking out git https://github.com/amit-salesforce/SFDXProject into C:\Program Files (x86)\Jenkins\workspace\Salesforce22_SFDC@script to read Jenkinsfile
    No credentials specified
    > git.exe rev-parse --is-inside-work-tree # timeout=10
    Fetching changes from the remote Git repository
    > git.exe config remote.origin.url https://github.com/amit-salesforce/SFDXProject # timeout=10
    Fetching upstream changes from https://github.com/amit-salesforce/SFDXProject
    > git.exe --version # timeout=10
    > git.exe fetch --tags --force --progress https://github.com/amit-salesforce/SFDXProject +refs/heads/*:refs/remotes/origin/*
    > git.exe rev-parse "refs/remotes/origin/master^{commit}" # timeout=10
    > git.exe rev-parse "refs/remotes/origin/origin/master^{commit}" # timeout=10
    Checking out Revision da509ec48c48efbf9571cbc2dbf2d6b21218c49f (refs/remotes/origin/master)
    > git.exe config core.sparsecheckout # timeout=10
    > git.exe checkout -f da509ec48c48efbf9571cbc2dbf2d6b21218c49f
    Commit message: "Update package.xml"
    > git.exe rev-list --no-walk da509ec48c48efbf9571cbc2dbf2d6b21218c49f # timeout=10
    Running in Durability level: MAX_SURVIVABILITY
    [Pipeline] Start of Pipeline
    [Pipeline] node
    Running on Jenkins in C:\Program Files (x86)\Jenkins\workspace\Salesforce22_SFDC
    [Pipeline] {
    [Pipeline] echo
    KEY IS
    [Pipeline] echo
    b711b818-2992-46ed-a8dd-f1ac66b0deaa
    [Pipeline] echo
    chirag.dhangar07@gmail.com
    [Pipeline] echo
    login.salesforce.com
    [Pipeline] echo
    3MVG9ZL0ppGP5UrAvVhi0FZQlTUSCnzOEt2sncSqazmXTiyyxi5v5dOhlWS6iCjKsC3_klGjxwCwYLdB68zSK
    [Pipeline] tool
    [Pipeline] }
    [Pipeline] // node
    [Pipeline] End of Pipeline
    ERROR: No tool named toolbelt found
    Finished: FAILURE

    ReplyDelete
    Replies
    1. You need to setup the toolbelt

      Delete
    2. Hi Amit,
      I am getting the same error. What exactly do we need to do to setup toolbelt. Can you share a help link for the same.

      Delete
    3. Install the Custom Tools Plugin into your Jenkins console, and create a custom tool that references the Salesforce CLI

      Delete
    4. This comment has been removed by the author.

      Delete
    5. I am also getting the same error. "Install the Custom Tools Plugin into your Jenkins console" i have already installed the Custom tools plugin. "and create a custom tool that references the Salesforce CLI" how to do this part?

      Delete
    6. Hi Amit,

      I have install the Custom Tools plugin.
      Now how to create a custom tool that references the Salesforce CLI?
      thanks in advance

      Delete
    7. I am assuming that you have already installed the Salesforce CLI on your machine. If not, then install it first using this link.
      https://developer.salesforce.com/tools/sfdxcli

      Go to Jenkins > Manage Jenkins > Global Tool Configuration
      Under the heading Custom Tool, click on "Custom tool configurations...". Click on "Add Custom Tool"
      Name - toolbelt
      Click on "Custom Tool Configuration"
      (for windows) Installation directory - C:\Program Files\sfdx\bin
      (for mac) Installation directory - /usr/local/bin

      Apply and Save

      Delete
    8. Thanks Amith. now I am getting another error like ' Could not find credentials entry with ID '362c307f-3224-445f-9ea8-7bc280c9eee6'. do you have any idea about this error?

      Delete
  6. Failed to connect to repository : Command "git.exe ls-remote -h https://github.com/amit-salesforce/SFDXProject HEAD" returned status code 128:
    stdout:
    stderr: fatal: unable to access 'https://github.com/amit-salesforce/SFDXProject/': SSL certificate problem: unable to get local issuer certificate

    ReplyDelete
  7. Validate Authorize an Org Using the JWT-Based Flow
    sfdx force:auth:jwt:grant --clientid {ADD_YOUR_CLIENT_ID} --jwtkeyfile server.key --username amit.salesforce21@gmail.com --instanceurl https://login.salesforce.com --setdefaultdevhubusername

    Where can I get this client id?

    ReplyDelete
    Replies
    1. Its the consumer key that you get when making a New Connected App in Salesforce

      Delete
  8. I get the following error when building the first time

    [Pipeline] }
    [Pipeline] // stage
    [Pipeline] withCredentials
    [Pipeline] // withCredentials
    [Pipeline] }
    [Pipeline] // node
    [Pipeline] End of Pipeline
    java.lang.NullPointerException
    at com.cloudbees.plugins.credentials.CredentialsProvider.findCredentialById(CredentialsProvider.java:874)
    at com.cloudbees.plugins.credentials.CredentialsProvider.findCredentialById(CredentialsProvider.java:852)
    at org.jenkinsci.plugins.credentialsbinding.MultiBinding.getCredentials(MultiBinding.java:151)
    at org.jenkinsci.plugins.credentialsbinding.impl.AbstractOnDiskBinding.bindSingle(AbstractOnDiskBinding.java:39)
    at org.jenkinsci.plugins.credentialsbinding.Binding.bind(Binding.java:150)
    at org.jenkinsci.plugins.credentialsbinding.impl.BindingStep$Execution2.doStart(BindingStep.java:134)
    at org.jenkinsci.plugins.workflow.steps.GeneralNonBlockingStepExecution.lambda$run$0(GeneralNonBlockingStepExecution.java:77)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
    Finished: FAILURE

    Anny suggestions what I can do?

    ReplyDelete
    Replies
    1. Is this issue solved? I am facing the same issue

      Delete
    2. You need to add Custom Tool "toolbelt" in Global Tool Configuration

      Delete
  9. Hi Amit,

    Sub: toolbelt setup issue.

    I did below steps, but getting exception. I didn't find any field as u mentioned in ur previous comments as "Installation directory".

    Global Tool Configuration -> Custom tool -> Custom tool installations:
    Custom tool Name: toolbelt
    Exported paths: C:\Program Files\sfdx\bin


    [Pipeline] tool
    [Pipeline] }
    [Pipeline] // node
    [Pipeline] End of Pipeline
    java.net.MalformedURLException: no protocol:
    at java.net.URL.(Unknown Source)
    at java.net.URL.(Unknown Source)
    at java.net.URL.(Unknown Source)
    at hudson.tools.ZipExtractionInstaller.performInstallation(ZipExtractionInstaller.java:83)
    at hudson.tools.InstallerTranslator.getToolHome(InstallerTranslator.java:69)
    at hudson.tools.ToolLocationNodeProperty.getToolHome(ToolLocationNodeProperty.java:109)
    at hudson.tools.ToolInstallation.translateFor(ToolInstallation.java:206)

    Please advise.

    Thanks.

    ReplyDelete
    Replies
    1. Is this solved? What is the solution? I am facing same issue.

      Delete
    2. Is this Solved ? I'm also facing the same issue

      Delete
  10. This comment has been removed by the author.

    ReplyDelete
  11. Hi Amit,

    I am getting this error!
    C:\Program Files (x86)\Jenkins\workspace\DevOrg_>"" force:auth:jwt:grant --clientid 3MVG9G9pzCUSkzZvL0eg5JOEbMZU6ogOA5ZNMyfnMLAKMQEalxWTeEsi_tZr5rUhkrvNDWXKIdmexm9VhFFUO --username bitbucket-integration@concret.io --jwtkeyfile "****" --setdefaultdevhubusername --instanceurl https://login.salesforce.com
    '""' is not recognized as an internal or external command,
    operable program or batch file.
    [Pipeline] error
    [Pipeline] }
    [Pipeline] // stage
    [Pipeline] }
    [Pipeline] // withCredentials
    [Pipeline] }
    [Pipeline] // node
    [Pipeline] End of Pipeline
    ERROR: hub org authorization failed
    Finished: FAILURE

    Can you please tell me that how to setup toolbelt, because I think I am getting this error because of toolbelt only?

    Thanks

    ReplyDelete
    Replies
    1. Hi vikas,
      I am getting same error .
      did you get any solution for that one?
      if you find any solution please let me know?

      Delete
    2. I'm also getting same issue.

      Delete
  12. HI All,
    I am facing an error :'""' is not recognized as an internal or external command,
    operable program or batch file.

    Any help is much appreciated.

    Thanks

    ReplyDelete
  13. Hi Amit,

    Thank you for your post.

    While I am implementing this method, I have few questions in mind and appreciate any response from you !! :)

    1. Using this method , can we integrate/deploy code, config as well as lightning components?
    2. Do you find any disadvantages of this method so far? With DX , which one is a recommended approach?

    Thanks!

    Kind regards,
    Milli

    ReplyDelete
  14. Hey Amit,

    Can Jenkins do the integration with Jira.

    ReplyDelete
  15. Subhash Beniwal20 July 2019 at 13:45

    Amit Chaudhary

    I am facing following issue to setup Jenkins, Please help me to resolve this issue.

    [Pipeline] }
    [Pipeline] // node
    [Pipeline] End of Pipeline
    java.net.MalformedURLException: no protocol:
    at java.net.URL.(Unknown Source)
    at java.net.URL.(Unknown Source)
    at java.net.URL.(Unknown Source)
    at hudson.tools.ZipExtractionInstaller.performInstallation(ZipExtractionInstaller.java:83)
    at hudson.tools.InstallerTranslator.getToolHome(InstallerTranslator.java:69)
    at hudson.tools.ToolLocationNodeProperty.getToolHome(ToolLocationNodeProperty.java:109)
    at hudson.tools.ToolInstallation.translateFor(ToolInstallation.java:206)
    at com.cloudbees.jenkins.plugins.customtools.CustomTool.forNode(CustomTool.java:163)
    at com.cloudbees.jenkins.plugins.customtools.CustomTool.forNode(CustomTool.java:65)
    at org.jenkinsci.plugins.workflow.steps.ToolStep$Execution.run(ToolStep.java:152)
    at org.jenkinsci.plugins.workflow.steps.ToolStep$Execution.run(ToolStep.java:133)
    at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
    Finished: FAILURE

    Thanks,
    Subhash

    ReplyDelete
  16. Hi Amit, how do i define RunSpecifiedTests in the deploy command and how do i mention those specified classes in the DX folder structure, like the way we use to add in build.properties file while deploying using ANT. Any help would be appreciated.
    Thanks

    ReplyDelete
  17. Hi Amit,

    I am still facing issue for toolbelt. I set the path as you mentioned in the Custom tool for toolbelt.
    Any help in this will be much appreciated.
    java.net.MalformedURLException: no protocol:
    at java.net.URL.(Unknown Source)
    at java.net.URL.(Unknown Source)
    at java.net.URL.(Unknown Source)
    at hudson.tools.ZipExtractionInstaller.performInstallation(ZipExtractionInstaller.java:83)
    at hudson.tools.InstallerTranslator.getToolHome(InstallerTranslator.java:69)
    at hudson.tools.ToolLocationNodeProperty.getToolHome(ToolLocationNodeProperty.java:109)
    at hudson.tools.ToolInstallation.translateFor(ToolInstallation.java:206)
    at com.cloudbees.jenkins.plugins.customtools.CustomTool.forNode(CustomTool.java:163)
    at com.cloudbees.jenkins.plugins.customtools.CustomTool.forNode(CustomTool.java:65)
    at org.jenkinsci.plugins.workflow.steps.ToolStep$Execution.run(ToolStep.java:152)
    at org.jenkinsci.plugins.workflow.steps.ToolStep$Execution.run(ToolStep.java:133)
    at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
    Finished: FAILURE

    Thanks!

    ReplyDelete
    Replies
    1. You need to add Custom Tool "toolbelt" in Global Tool Configuration

      Delete
    2. Remove the auto installer for Custom Tools and SFDX Path in extracted path.

      Delete
    3. Hi Amit/Samrat,
      Thanks for the post. Can you please elaborate the steps for removing Auto Installer. I'm facing the same issue.
      Would really appriciate your reply as it is urgnet for me.

      Delete
    4. This comment has been removed by the author.

      Delete
    5. In Custom Tool, remove the Auto Installer under "toolbelt" and add the SFDX CLI path like below.
      (Example: "C:\Program Files\Salesforce CLI\bin")

      Delete
  18. Hi Amit,

    Thanks for the detailed explanation for CI using SFDX.
    I was trying build(as part of a trailhead module challenge) CI pipeline using Gitlab, i have done everything,
    1- Setup DevHub , 2-Generated certificate using OpenSSL, created Connected app and gave proper profile , 4- Added certificate to Connected app.

    and then granted access _
    sfdx force:auth:jwt:grant --clientid 3MVG9GYWKbMgBvbxdetjB7QiPpcjZ.TcXna3q0e2EKVaVx8v.AtyFsvp0R_iqCOy17niCM.Iw04h8.KVIbTkX --jwtkeyfile ./JWT/server.key --username prakash.sfdc26@curious-bear-mp8v4r.com --setdefaultdevhubusername

    and got "Successfully authorized prakash.sfdc26@curious-bear-mp8v4r.com with org ID 00D4T000000G0vWUAS" message aswell.

    but my challange is failing with error "We could not confirm that you authenticated to your Dev Hub org using JWT-based auth flow with the 'DreamHouse_demo' Connected App".

    I am repeating the above steps from last 2 days, still no luck. I am sure i am missing something minor, If you have any suggestion/idea why its failing, very much appreciate that.

    Thanks, Waiting for you reply,
    Prakash

    ReplyDelete
  19. Hi Amit, That was a good knowledge sharing about jenkins.
    I had followed all of your steps and i'm getting the below Error as :
    ERROR running force:mdapi:deploy: The specified path [manifest/.] does not exist
    C:\Program Files (x86)\Jenkins\workspace\SEGRDHT_>"C:\Program Files\Salesforce CLI\bin\sfdx" force:mdapi:deploy -d manifest/. -u demogmail@gmail.com
    [Pipeline] }
    [Pipeline] // stage
    [Pipeline] }
    [Pipeline] // withCredentials
    [Pipeline] }
    [Pipeline] // node
    [Pipeline] End of Pipeline
    ERROR: script returned exit code 1
    Finished: FAILURE


    Can you please help me out on this what can be the issue.

    ReplyDelete
  20. Failed to connect to repository : Error performing command: git.exe ls-remote -h https://github.com/guptaaalokkumar/SFDXProject.git HEAD
    getting above error

    ReplyDelete
  21. Hey thanks for this amazing post! Thank you so much for sharing the good post, I appreciate your hard work.Keep blogging.
    DevOps Training in Electronic City

    ReplyDelete
  22. Thank you for posting the valuable information.
    DevOps Online Training

    ReplyDelete