Ishmael Not Receiving the Covenant?

In Romans 4, Paul addresses Genesis 15 and Genesis 17. He seems to take very specific detail in both the comment in verse 6 of 15 and the act of circumcision. I think it's pretty clear that even though Ishmael is being circumcised, he's not receiving the covenant. And so Abraham was given righteousness as a gift because of his faith.