Building Augmented Reality Experiences with ARCore

ARCore gives Android apps a reliable way to combine camera imagery with 3D content, spatial anchors and device-pose tracking. Instead of treating the screen as a flat display, the app can detect surfaces, estimate lighting and place objects that appear to rest in the user’s environment. The result is spatial computing that remains interactive as the phone moves.

A dependable augmented reality experience begins with device support, clear fallbacks and a small interaction that works quickly. ARCore relies on Google Play Services for AR, so the Android device, installed system services and app version must all be compatible. This matters in Australia, where Android is widely used but supported handset models, screen sizes and performance levels vary considerably.

Prepare the Android project

Start with a normal Android app module, set the minimum SDK to 24 or higher, and add the SDK from Google’s Maven repository. Pin a known stable version so builds remain repeatable:

dependencies {
    implementation("com.google.ar:core:1.47.0")
}

The version above is an example pin; use the current supported release when starting a new project. Check the release notes before upgrading, and rely on the official Google Play Services for AR installation flow where possible.

The manifest must declare camera access and the AR hardware feature. A required feature limits installation to compatible phones, while an optional feature permits a non-AR fallback:

<uses-permission android:name="android.permission.CAMERA" />
<uses-feature
    android:name="android.hardware.camera.ar"
    android:required="false" />

Change required to true for an AR-only release. Keep the fallback meaningful if the app is distributed broadly.

Configure the camera and ARCore session

Create one ARCore Session and reuse it while the activity is active. Configure horizontal and vertical plane detection when the scene needs floors and walls, and request ambient light data so rendered objects can match the surroundings:

val arSession = Session(this)
val config = Config(arSession).apply {
    planeFindingMode = PlaneFindingMode.HORIZONTAL_AND_VERTICAL
    lightEstimationMode = LightEstimationMode.AMBIENT_INTENSITY
}
arSession.configure(config)

Before creating the session, call ArCoreApk.getInstance().checkAvailability(context). Route unsupported or outdated states to installation and update guidance. A session that is recreated carelessly can interrupt tracking or waste camera resources.

Request CAMERA permission with the Activity Result API. Explain why the camera is needed before showing the system prompt, then call arSession.resume() only after permission is granted. If permission is denied, present a static preview or another useful screen rather than leaving a blank camera view.

Call session.update() once per rendered frame and inspect frame.camera.trackingState. Pause and resume the session with the activity, handle camera-unavailable errors, and release it in onDestroy. Display tracking guidance when the camera is still initialising, because taps made before pose estimation are unlikely to land accurately.

Place and animate virtual objects

Plane detection provides candidate surfaces for placement. A tap can be converted into a raycast, filtered to a detected plane and attached to an anchor:

val hit = frame.hitTest(tapX, tapY).firstOrNull {
    it.trackable is Plane
}
val anchor = hit?.createAnchor()

Store the anchor and read its pose while rendering. Anchors are useful for furniture previews and instructions because the content stays aligned with a recognised location as the user moves.

ARCore supplies tracking data rather than a complete renderer; combine it with OpenGL, Filament or a maintained SceneView library. Give the renderer the camera texture through arSession.setCameraTextureName(textureId), then draw the camera background before virtual objects. Use Android gesture handling for selection, dragging, rotation and scaling, but convert touch input into local anchor coordinates. Avoid creating an unrestricted anchor for every tap, because unmanaged anchors increase processing and tracking work. For a small catalogue, remove an old anchor before adding its replacement and limit simultaneous objects to what the device can render smoothly.

Beyond simple placement, ARCore can provide depth-based occlusion, indoor understanding and geospatial features on supported devices and in supported locations. These capabilities need explicit checks because they are not equally available across phones or regions. A plane-anchored mode should remain usable when an advanced mode is unavailable.

Design for Australian conditions and users

Australian retail and tourism trials often move between bright outdoor streets, covered markets and dim interiors. A placement demo near Sydney’s harbour or around Melbourne’s laneways may encounter sun glare, reflective glass and blank walls that give the tracker little detail. At places such as Queen Victoria Market, signage, crowds and changing light can also make objects difficult to distinguish, so outdoor testing is essential.

Environmental tracking affects usability as well as rendering. Use the ambient-light estimate to adjust brightness, add a clear shadow or ground cue beneath virtual objects, and offer an alignment reticle rather than assuming a tap will work. Strong Australian sunlight, indoor air-conditioning and variable regional weather all change the visual conditions. Give users a simple reset action and a clear message when tracking is paused instead of letting confusing imagery continue.

Handle data deliberately. ARCore’s world tracking can run locally, but your app may still collect location, analytics identifiers or uploaded images. For organisations covered by the Privacy Act 1988, the Australian Privacy Principles require transparent handling of personal information, including clear purposes, access and deletion controls. Camera permission is not permission to retain or transmit images, and precise location should be off by default unless the experience genuinely needs it.

Test, optimise and release responsibly

Test on several physical ARCore-certified Android devices, not only an emulator. Include lower-powered phones, different screen ratios and devices without depth support. Exercise the full permission flow, interrupted calls, backgrounding, low storage and loss of internet access. Measure frame rate, thermal behaviour and battery use during sustained sessions, because Australian users often use navigation or shopping apps on long outdoor outings.

Before release, run the Android vitals and Google Play pre-launch reports, then verify that the store listing accurately describes the AR requirement. The Play Console device catalogue can confirm which models are eligible for an AR-required package. If a companion watch workflow needs Garmin account access, isolate that authentication in a separate module and never send ARCore camera frames or pose data to an unrelated account service.

Keep privacy information and data-safety declarations aligned with actual behaviour, including third-party analytics. Under the Australian Consumer Law, paid experiences and performance claims must not be misleading. A small, supported feature set with reliable fallback behaviour generally produces a better release than a broad feature list that fails on common regional devices.