Android builds from Hudson are challenging in a completely different way than the iPhone builds. The challenges are mainly centered around updating the Android manifest file (which is owned by the Android development team, not the team doing the builds). So we need to dynamically load and change this file from version control before we build.
1. The svn control and tagging from Hudson is handled in the same way as the iPhone builds (see Hudson iPhone builds pt. 1)
2. Since we need to update the Android manifest file for each build, we need a build.xml that is different than a development build.xml and can handle the manifest update steps. The way we're handling this now is by taking the developer build.xml (which was originally created by the build team) and adding or ANT replace targets to it. We then overwrite the existing build.xml with this one every time.
There's a possible issue here in that the developers could alter the build.xml, and we'd need to handle it dynamically also (do the same thing that we're doing for the manifest). In that case it should be possible to create an ANT build.xml to alter the Android build.xml every run. But for now our first step is to just copy over the altered build.xml and overwrite the dev one:
xcopy /y/q "D:\Property_files\Android_trunk\build.bak4" D:\Hudsonb\jobs\Android-trunk\workspace\android_trunk\build.xml
3. The local.properties file is critical for these builds and will also vary according to build machine. So next step is to copy that file over:
xcopy /y/q "D:\Property_files\Android_trunk\local.properties" D:\Hudsonb\jobs\Android-trunk\workspace\android_trunk
4. For Android builds it's critical that the version.code be incremented each build. The new build must have a higher value for version.code in the Android manifest, or loading the app should fail for the user. So the next step in the build is to copy over a build.number file that will then get incremented by an ANT task. This file follows standard ANT increment format (will go in detail in another post):
xcopy /y/q "D:\Property_files\Android_trunk\build.number" D:\Hudsonb\jobs\Android-trunk\workspace\android_trunk
5. We always do an ant clean task before build:
setlocal
set JAVA_HOME=D:\ProgramFiles\Java\jdk1.6.0_11
set ANDROID_HOME=D:\ProgramFiles\Java\android-sdk-windows
set ANT_HOME=D:\ProgramFiles\Ant\apache-ant-1.7.1\apache-ant-1.7.1
cd D:\Hudsonb\jobs\Android-trunk\workspace\android_trunk
ant clean
6. We then originally ran a gamut of replace tasks on the manifest file. Dev have revised the procedure so that the only extra task we need to do is update that version.code value:
setlocal
set JAVA_HOME=D:\ProgramFiles\Java\jdk1.6.0_11
set ANDROID_HOME=D:\ProgramFiles\Java\android-sdk-windows
set ANT_HOME=D:\ProgramFiles\Ant\apache-ant-1.7.1\apache-ant-1.7.1
cd D:\Hudsonb\jobs\Android-trunk\workspace\android_trunk
ant updateversioncode
7. Then run release to create the Android build:
setlocal
set JAVA_HOME=D:\ProgramFiles\Java\jdk1.6.0_11
set JAVA_OPTS=-Xms64m -Xmx256m
set ANDROID_HOME=D:\ProgramFiles\Java\android-sdk-windows
set ANT_HOME=D:\ProgramFiles\Ant\apache-ant-1.7.1\apache-ant-1.7.1
set ANT_OPTS=-Xms64m -Xmx256m
cd D:\Hudsonb\jobs\Android-trunk\workspace\android_trunk
ant release
8. We then sign the release. Since the build machines are 'locked down' we pass these values in through the build itself:
setlocal
set JAVA_HOME=D:\ProgramFiles\Java\jdk1.6.0_11
set ANDROID_HOME=D:\ProgramFiles\Java\android-sdk-windows
set ANT_HOME=D:\ProgramFiles\Ant\apache-ant-1.7.1\apache-ant-1.7.1
cd D:\ProgramFiles\Java\jdk1.6.0_11\jre\bin
jarsigner -keystore D:\ProgramFiles\Java\jdk1.6.0_11\bin\blah.keystore -storepass blah2 -keypass blah2 -signedjar D:\Hudsonb\jobs\Android-trunk\workspace\android_trunk\bin\blah3pre.apk D:\Hudsonb\jobs\Android-trunk\workspace\android_trunk\bin\blah3-unsigned.apk blah_key
9. We then need to do a zipalign on the output, this is required for Android deliveries:
setlocal
set JAVA_HOME=D:\ProgramFiles\Java\jdk1.6.0_11
set ANDROID_HOME=D:\ProgramFiles\Java\android-sdk-windows
set ANT_HOME=D:\ProgramFiles\Ant\apache-ant-1.7.1\apache-ant-1.7.1
cd D:\ProgramFiles\Java\android-sdk-windows\tools
zipalign -f -v 4 D:\Hudsonb\jobs\Android-trunk\workspace\android_trunk\bin\blahpre.apk D:\Hudsonb\jobs\Android-trunk\workspace\android_trunk\bin\blah.apk
10. Since we incremented the build.number file we need to move it back so that the next build increments again:
xcopy /y/q "D:\Hudsonb\jobs\Android-trunk\workspace\android_trunk\build.number" D:\Property_files\Android_trunk
Next post, diving into the details of the manifest replace steps and how Android uses the properties files
Friday, May 28, 2010
Thursday, May 27, 2010
Hudson iPhone builds part 3
I ran into some issues with delivering the completed iPhone zip from the Mac, so I ended up creating some separate deploy jobs that take care of it.
1. In the trunk builds I have the 'Build other projects' setting checked and call the deploy job. For branch builds I use the 'Trigger parameterized build on other projects' and choose 'Current build parameters', the reason being that I need to send the branch value to the deploy job.
2. I have the 'Post-build Actions' - 'Archive the artifacts' set for the build job. This takes the zip created by the Mac and archives it locally on the Hudson server. This is critical to the issue with Mac delivery.
3. For the configuration of the deploy job, I have the 'Advanced Project Options' - 'Use custom workspace' setting checked, and list the builds folder of the parent Hudson job that initiates this deploy job. It will find the zip that was archived to Hudson in the previous step. This means that each build needs its own deploy job, which doesn't scale perfectly, but since this is limited to Mac builds it's been working out OK
4. The next step in the deploy job is 'Publish artifacts to SCP Repository' and send it from the Hudson archive location.
5. There is then a job called to delete the zip from the Hudson server. This is because the archive job retains at a minimum two zips (last two successful), and we don't want the deploy job to deliver two. The delete job is called with 'Build other projects'
6. For the delete job the 'Use custom workspace' is using the build home from the original build job but handles the deletes for all jobs. This is one line that's called from the job:
del /s/q \hudson-iPhone*.zip
That's it for the build and delivery, maybe will dive into the iPhone code signing next
1. In the trunk builds I have the 'Build other projects' setting checked and call the deploy job. For branch builds I use the 'Trigger parameterized build on other projects' and choose 'Current build parameters', the reason being that I need to send the branch value to the deploy job.
2. I have the 'Post-build Actions' - 'Archive the artifacts' set for the build job. This takes the zip created by the Mac and archives it locally on the Hudson server. This is critical to the issue with Mac delivery.
3. For the configuration of the deploy job, I have the 'Advanced Project Options' - 'Use custom workspace' setting checked, and list the builds folder of the parent Hudson job that initiates this deploy job. It will find the zip that was archived to Hudson in the previous step. This means that each build needs its own deploy job, which doesn't scale perfectly, but since this is limited to Mac builds it's been working out OK
4. The next step in the deploy job is 'Publish artifacts to SCP Repository' and send it from the Hudson archive location.
5. There is then a job called to delete the zip from the Hudson server. This is because the archive job retains at a minimum two zips (last two successful), and we don't want the deploy job to deliver two. The delete job is called with 'Build other projects'
6. For the delete job the 'Use custom workspace' is using the build home from the original build job but handles the deletes for all jobs. This is one line that's called from the job:
del /s/q \hudson-iPhone*.zip
That's it for the build and delivery, maybe will dive into the iPhone code signing next
Wednesday, May 26, 2010
Hudson iPhone builds part 2
The iPhone code we're building against is located in a subversion repository, so to bring it to the Mac build machine we just use Hudson's source code management link to svn within the build.
Once the code has been brought over, here are our build steps:
1. Tag the code in svn. I tried some of the integrated Hudson tools for svn tagging but ended up using a straight svn copy from the shell, like:
cd /users/administrator/Hudson/workspace/
svn copy http://somesvnserver/svn/Code/iPhone http://somesvnserver/svn/Code/tags/$BUILD_TAG -m "Create -iPhone tag - trunk" --username blah --password blah
2. For the simulator builds we jump right to the build step. But for device builds there's a need to do a valid code signing, and that means unlocking the keychain from the command line. So for device builds our second step looks like this:
cd /Library/Keychains
security unlock -p password login.keychain
cd /users/administrator/Hudson/workspace/
xcodebuild clean -alltargets -configuration Release build SDKROOT=/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS3.1.3.sdk TARGETED_DEVICE_FAMILY=iPhone
3. For internal builds we copy in the provisioning file with each build:
cp "/users/administrator/Hudson/workspace/Certs and Profiles/iphone.provision" /users/administrator/Hudson/workspace/release-iphoneos
4. The next step removes the dSYM directory from the build. There's a way to do this from the xcode build call itself, just doing this temporarily to remove those folders:
cd /users/administrator/Hudson/workspace/Release-iphoneos
rm -r iphone.app.dSYM/Contents/Resources/DWARF
rm -r iphone.app.dSYM/Contents/Resources
rm -r iphone.app.dSYM/Contents
rm -r iphone.app.dSYM
5. Then a step to zip up the contents of the folder so it can be delivered:
cd /users/administrator/Hudson/workspace/build
ditto -c -k --sequesterRsrc --keepParent release-iphoneos $BUILD_TAG.zip
Next post, how the zip is delivered
Once the code has been brought over, here are our build steps:
1. Tag the code in svn. I tried some of the integrated Hudson tools for svn tagging but ended up using a straight svn copy from the shell, like:
cd /users/administrator/Hudson/workspace/
svn copy http://somesvnserver/svn/Code/iPhone http://somesvnserver/svn/Code/tags/$BUILD_TAG -m "Create -iPhone tag - trunk" --username blah --password blah
2. For the simulator builds we jump right to the build step. But for device builds there's a need to do a valid code signing, and that means unlocking the keychain from the command line. So for device builds our second step looks like this:
cd /Library/Keychains
security unlock -p password login.keychain
cd /users/administrator/Hudson/workspace/
xcodebuild clean -alltargets -configuration Release build SDKROOT=/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS3.1.3.sdk TARGETED_DEVICE_FAMILY=iPhone
3. For internal builds we copy in the provisioning file with each build:
cp "/users/administrator/Hudson/workspace/Certs and Profiles/iphone.provision" /users/administrator/Hudson/workspace/release-iphoneos
4. The next step removes the dSYM directory from the build. There's a way to do this from the xcode build call itself, just doing this temporarily to remove those folders:
cd /users/administrator/Hudson/workspace/Release-iphoneos
rm -r iphone.app.dSYM/Contents/Resources/DWARF
rm -r iphone.app.dSYM/Contents/Resources
rm -r iphone.app.dSYM/Contents
rm -r iphone.app.dSYM
5. Then a step to zip up the contents of the folder so it can be delivered:
cd /users/administrator/Hudson/workspace/build
ditto -c -k --sequesterRsrc --keepParent release-iphoneos $BUILD_TAG.zip
Next post, how the zip is delivered
Hudson iPhone builds part 1
The first big step we had to take to get iPhone building was to connect up our Mac to the Hudson server. The connection would have to get around password prompting, which does not work out of the box on a Mac. So after much investigation here are the steps we had to take:
1. For non-interactive builds and deploys, SSH has to be configured so it does not prompt for passwords. It seems that by default Mac OSX 10.6.2 comes with password prompt turned on
2. If the Finder menu is open on the top of the Mac window, click on Go - Go to Folder
3. Type in /private/etc and click Go
4. Right click on the sshd_config file and choose Copy. Right click on the desktop and Paste Item
5. Open the file with TextEdit the file. There are 2 key settings. Remove the comment (#) from the lines so they are active:
PasswordAuthentication yes
ChallengeResponseAuthentication no
6. Save the file on close of TextEdit
7. Right click and copy the file, and paste back to the /private/etc folder. Overwrite the existing
1. For non-interactive builds and deploys, SSH has to be configured so it does not prompt for passwords. It seems that by default Mac OSX 10.6.2 comes with password prompt turned on
2. If the Finder menu is open on the top of the Mac window, click on Go - Go to Folder
3. Type in /private/etc and click Go
4. Right click on the sshd_config file and choose Copy. Right click on the desktop and Paste Item
5. Open the file with TextEdit the file. There are 2 key settings. Remove the comment (#) from the lines so they are active:
PasswordAuthentication yes
ChallengeResponseAuthentication no
6. Save the file on close of TextEdit
7. Right click and copy the file, and paste back to the /private/etc folder. Overwrite the existing
Subscribe to:
Posts (Atom)
