I don't quite understand why you'd want to refernce/unreference the Office library from your VBA project, however the following are (untested) procedures to add and remove the references
Code:
Public Sub addRefToOffice()
On Error Resume Next
Application.VBE.ActiveVBProject.References.AddFromFile "C:\Program Files\Microsoft Office\Office11\MSO11.DLL"
End Sub
Public Sub removeRefToOffice()
Dim ref As Object
On Error Resume Next
Set ref = Application.VBE.ActiveVBProject.References("Office")
Application.VBE.ActiveVBProject.References.Remove ref
End Sub
Hope that gets you started