A retractable handle assembly for a luggage includes an outer pipe, an intermediate pipe, an inner pipe, and a control handle. Each of the inner pipe and the intermediate pipe has a bottom end provided with a support seat and a locking device. The support seat includes a housing and an end cap. The locking device includes at least one press member, at least one locking pin, and at least one positioning pin. Thus, the inner pipe and the intermediate pipe are positioned exactly and stably so that the inner pipe and the intermediate pipe will not slip when the retractable handle assembly is folded and expanded, thereby facilitating a user operating the retractable handle assembly.