Can I plug 125v to 120v?
Moreover, can I use a 125v cord on a 120v?
If so yes 125V means that is the maximum voltage the cord can stand at its rated amperage, so if your voltage is 120V or less then your cord can actually withstand a slightly higher amperage. You can plug your extension cord into any voltage up to this value: 125V.
Simply so, can I plug 125v to 110v?
In the US, 110V isn't a standard voltage. Almost all receptacles supply 120 to 125V. But appliances are usually designed to work with a range of voltages. 110V is only 8% lower than 120V, so even if the voltage is only 110V, say at the end of a long line, any 120V appliance should work fine.
You can always use a power cord that is rated at a higher voltage than the maximum required by your setup. Your original cord was rated at 10A, 125V. The one you found is also rated at 10A, and it will handes 250V, twice the voltage it will actually encounter. Bottom line - You should be good to go with the new cord.