This kind of broken promise does not hold water because had something happened to your brother, your family and relatives will compound the blame on you. They will not understand that you promised him o.

Thank God you thought of his safety first. Apologies will make up for everything later.

Such a predicament is not something that one should handle on his own at all. Thank God for your quick action.