What do you mean when you say "working with numerical fields is quicker than text fields"?
What are you trying to do? Generally you use number data types only when doing calculations. If you are storing a number that will not be used in calculations, such as phone numbers and zip codes, you use text. If you are doing any calculations, except for currency, you use number, and currency for cash.
If you use a number field where you want a text field, then you can end up with formatting issues and erroneous equations when doing queries.
I don't ever recall seeing that numbers work faster than text, except perhaps in this case where your colleagues are converting the data (presumably in code [re-typing a variable]) and that will always add time to a function or subroutine - but not much usually, unless it is a big batch job. I think otherwise the only thing that would determine the speed of the operation is the AMOUNT of data stored in the field, and not necessarily the data type. If you are using 9 character values in your text field, set the field size to 9, and this will speed up the operations in the aggregate.
Did that help any?