Archive for the ‘Programming’ Category

HTML 5 Audio playlists on iOS and Android

Posted: October 10, 2012 by Ash Mishra in Html 5, Mobile, Programming

Here is working code tested on iOS 5.x+ and Android 2.3+, using the HTML 5 Audio tag and jQuery, to play two mp3 tracks back to back.

For both Android and iOS, autoplay works (wifi on iOS 6 – not tested with 3G)

On iOS, there is even an extra: audio tracks can continue to be played with Safari in the background, and even with the screen off. Previous, Next, Play, Pause controls work too!

On Android, if the screen is off or the web page isn’t visible, the audio unfortunately won’t play the next track. Returning to the browser will start it off again though.

<!DOCTYPE html>
<html>

<head>
<meta name = "viewport" content = "width = device-width">
<script src="http://code.jquery.com/jquery-1.8.2.min.js"></script>

</head>
<body>
The built-in HTML 5 audio player is shown for debug purposes. You should control the audio with the buttons below.
<audio autoplay controls="controls" id="player" x-webkit-airplay="allow" style="opacity: 1; top: 0px; left: 0px; z-index: 1; margin-bottom: 10px; display: block; width: 250px;">
 <source id="audio_source" src="audio/starbuck.mp3" type="audio/mpeg" />
</audio>
<button onClick="playNow();">Play</button>
<button onClick="pause();">Pause</button>
<button onClick="playNext();">Next</button>
<script type="text/javascript" language="javascript">
 $(function() {
 $('#player').bind('ended', playerEnded);
 $('#player').bind('canplay', playerLoaded);
 })

 function playerEnded(e) {
 // window.alert('player ended');
 if(!e) { e = window.event; }
 playNext();
 }

 function playNow() {
 $('#player').get(0).play();
 }

 function pause() {
 $('#player').get(0).pause();
 }

 function playNext() {
 if (window.HTMLAudioElement) {
 // window.alert('playing next');
 $('#audio_source').attr("src", "audio/guacamole.mp3").detach().appendTo($('#player'));
 $('#player').get(0).load();
 }
 }

 function playerLoaded() {
 // window.alert('player loaded');
 $('#player').get(0).play();
 }

</script>
</body>
</html>

Sensor Fusion Redux

Posted: January 2, 2012 by Mike Ilich in Design, Programming

In the previous article, we discussed the polling of individual gyroscope axis values for driving the orientation of an object model in virtual 3D space. As a review, the basic structure for developing a sensor-based UI for 3D navigation is entirely driven by the Core Motion library in iOS, for which the CMMotionManager class is instantiated as a direct interface to the onboard accelerometer and gyroscope hardware in iPhone, iTouch and iPad devices (gyroscope available as of iPhone 4, iTouch 4G and iPad 1G). Sample code to initialize the CMMotionManager is provided below:

   (CMTestViewController.h, in the private interface variable declaration area)
      CMMotionManager *motionManager;
   (CMTestViewController.m, in viewDidLoad or similar setup)
      motionManager = [[CMMotionManager alloc] init];
      motionManager.accelerometerUpdateInterval = 0.01;
      motionManager.deviceMotionUpdateInterval = 0.01;
      [motionManager startDeviceMotionUpdates];

At this point, the motionManager variable can be polled at any point in your code to retrieve a CMDeviceMotion structure, describing the physical motion of the device in discrete intervals defined by the update interval property described above. The accelerometerUpdateInterval and deviceMotionUpdateInterval represent the interval (in seconds) for providing accelerometer and device-motion updates to the block handler, respectively. The six degrees of physical motion can be described by accelerometer values for each independent axis (X, Y, Z) as well as the overall attitude of the device, derived from the embedded “attitude” class variable that can be retrieved through:

      motionManager.deviceMotion.attitude

This variable contains properties such as “.yaw”, “.pitch” and “.roll”, which we previously used to describe rotation of the device along its Z, X and Y axes respectively. In addition, “attitude” provides a property named “rotationMatrix” that provides a full 3×3 matrix describing the Euler rotations that are required to translate a rigid body in 3D Euclidian space. The Euler angles represent a series of aggregate rotations, each around a single axis, that define the orientation of a point of reference in a three dimensional coordinate system. Therefore, the rotation matrix derived from the gyroscope attitude can be described as the product of three elemental rotation matrices, each representing a single axis. As mentioned in the previous post, the order in which these rotations are applied can significantly affect the overall translation of a point in 3D space, consisting of yaw, pitch and roll.

In addition, these operations result in the introduction of Gimbal lock, or the loss of one degree of freedom when two out of the three rotation axes in 3D Euclidian space are oriented in parallel and the system becomes locked to rotation in 2D space. We observed this phenomenon as either the gyroscope X or Y axis was rotated in parallel with the Z axis, effectively locking rotation of the remaining axis. In order to counteract this effect, a static fourth axis is added by embedding the attitude.rotationMatrix in a 4×4 Matrix with an identity vector for the fourth dimension. Note that a model view matrix is defined for the object model in 3D space and subsequently multiplied by the rotation matrix to complete the transformation. The model view matrix is distinct from the projection matrix, which defines the camera angle, aspect ratio and near/far values for Z offset.

Here is an example of how this might be arranged in the GLKit library, exclusive to iOS 5.0:

      CMDeviceMotion *deviceMotion = motionManager.deviceMotion; 
      CMAttitude *attitude = deviceMotion.attitude;
      rotation = attitude.rotationMatrix;
      GLKMatrix4 rotationMatrix = GLKMatrix4Make(rotation.m11, rotation.m21, rotation.m31, 0, rotation.m12, rotation.m22, rotation.m32, 0, rotation.m13, rotation.m23, rotation.m33, 0, 0, 0, 0, 1);
      modelViewMatrix = GLKMatrix4Multiply(modelViewMatrix, rotationMatrix);

From this point, the modelViewMatrix can be scaled, rotated by a gravity correction factor or applied directly to the transform property of either an object model or skybox effect, inherent to the GLKit API that was included in the iOS 5.0 release to simplify OpenGL ES by providing a usable template and higher level function calls for common 3D modeling elements. By replacing the aggregate yaw, pitch and roll rotation operations with a full rotation matrix accounting for the Euler angles, we have resolved any further ambiguity in model view orientation as well as the incidence of Gimbal lock when taking into account the azimuth in 3D space.

2011 Macbook Air for iOS Developers

Posted: July 28, 2011 by Ash Mishra in Productivity, Programming, Review
Tags: ,
New MacBook Airs are here!

The reviews have started pouring in for the updated Core i5/i7 Macbook Airs released in July 2011, and they have been unanimously acclaiming the ultra-portable laptop / notebook. There’s no question they are a leap in performance from the prior models, and they are now powerful enough to be considered tools for more than the casual user.  I even think they are good enough to be considered by media professionals and developers.

I decided to pick up the 13″ Macbook Air to see if it could replace or be a secondary machine in use for iOS development.  I originally ordered the  top i7/256 model, but returned it in favour of the more affordable i5/128 which in my experience (so far) has been more than sufficient.  I have a 2007 17″ Macbook Pro (2.4 c2d / 4gb / 500gb HD / 1920 x 1200) that I have been using for my primary development. I have enjoyed using the 17″ screen for iPad development – particularly being able to use the simulator at full scale, while having XCode on the same screen.

(more…)

iOS Sensor Fusion

Posted: July 10, 2011 by Mike Ilich in Design, Programming

One might suggest that we’re approaching the golden age of Human Factors design with the introduction of peripheral sensory input that has been inherent in many smartphones for a number of years now. While multi-touch LCD screens have eliminated the need for a separate keyboard or pointing device, paving the way for modern tablets, screen real estate became the limiting factor with on-screen keyboards and the dreaded dual-thumbstick overlay for navigating 3D graphic environments. Built-in sensors such as accelerometers, gyroscopes and digital compasses have only recently seen extensive use as part of the overall UX design, offering physical motion and device orientation as viable UI controls.

(more…)

VBO (Vertex Buffer Object) Limitations on iOS Devices

Posted: July 10, 2011 by Kearwood "Kip" Gilbert in Programming
Tags: , , ,

With the iPhone 3GS, the iOS platform gained programmable shader support through the PowerVR SGX 535.  Accompanied with an increase from 2 to 8 texture units, artists and developers have limitless creative expression by using any material in their 3d scenes that they can describe in GLSL.  The unsung hero of these upgraded GPU’s are their “real” Vertex Buffer Objects (VBO).

Vertex Buffer Objects allow the vertex level data to be moved from slower main memory to memory that can be accessed more rapidly by the GPU.  Instead of making a separate GL call to render each polygon, the VBO’s can be initialized when a model is loaded and a single GL call instructs the GPU to iterate through the vertices and rendering polygons is performed by the GPU.  Even though the new IOS platforms have a UMM (Unified Memory Model) where the CPU and GPU share the same ram chips and bus, utilizing VBO’s frees up the ARM CPU core during rendering for other game pipeline tasks such as physics calculation, AI, and audio.

(more…)

iOS ALAsset image metadata

Posted: July 10, 2011 by Ash Mishra in Programming

iOS provides a built-in Assets Library, which stores media that you have sycned with iTunes, or stored in your Camera roll. These are usually photos or videos.

The main iOS classes for working with media Assets are the following:

  1. ALAssetsLibrary – which provides access to Photo Groups by category type. (ALAssetsGroupLibrary, ALAssetsGroupAlbum, ALAssetsGroupEvent, ALAssetsGroupFaces, ALAssetsGroupSavedPhotos, ALAssetsGroupAll)
  2.  After picking an ALAssetsLibrary, you can enumerate through the enclosed ALAssetsGroup objects
  3. An ALAssetsGroup object holds ALAssets, which are a wrapper representing a single image or video.

Other posts cover the details on using enumerators and blocks for getting a set of ALAssets, so I will discuss retrieving metadata.  This can be done in two ways:

  1. Call [asset metadata] which returns a dictionary containing metadata for an asset, including most of its EXIF content. When I tried this call, it did indeed work, but you may have some memory issues to deal with unless you ensure the calls are single-threaded.
  2. Use CGImageSourceCopyPropertiesAtIndex to retrieve individual properties from a CGImageSourceRef; this approach allows you to specify exact properties. See example code after the break below on retrieving a property from an ALAsset using this approach.
My tests for fetching metadata show that while option 2 is more code, it is several magnitudes faster than the much simpler [asset metadata] call in option 1.

As an aside, creating a CGImageSourceRef from an ALAssets is easy when it is a jpeg.  However, if you are dealing with other types of media formats, you will get an error when creating a CGImageSourceRef without a source hint identifying the type of image; this is done by passing in the ALAssetRepresentation UTI.

Two IF statements to Avoid

Posted: September 13, 2010 by Ash Mishra in Design, Programming

After 15 years of writing software, I’ve accumulated several thoughts on software design.

I didn’t go to school primarily for Software engineering, and I haven’t spent as much time reading heavy books on the subject as I would like.  Most of my career comes from hands-on experience, and learning from both successes and past reflection.  Indeed the phrase Design Pattern was a foreign one to me until a decade ago when I started realising some of what I had been practicising had terms coined for them.

Through experience, I’ve gravitated to several favourite patterns and key concepts.  On a new project these are the MVC (Model-View-Controller) pattern, the DRY (don’t repeat yourself) principle, the use of Convention over Configuration, the use of reflection, protocols, state engines, recursion, and remote configuration.

Sounds a little like the kitchen sink – doesn’t it?

But sophisticated maintainable software almost completely relies on design.  While there are many facets to design, one that I focus on heavily on is how reusable code is.

I hesitate to say this can be achieved by reading design pattern books, or the use of technical skills, as several times I have made the mistake of repeating these errors.

(more…)